List users logged in - need way to split fields - perl?

[Geoff Wild]

I run the following command (it grabs users logged in to servers via the security system we use)

boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\* |cat -v


Output is an extremely long line, here's part of it:


DATA=CDRP:fapirwan^Bsvr607:pts/4^B^B1180963725^B^BCDRP_PROD:cdrbatch^Bsvr607:pts/5^B^B1180965178^B^BCISCO:kfulford^Btmonprod01:pts/1^B^B1180015709^B^BCISCO:kfulford^Btmonprod01:pts/2^B^B1180038015^B^BCORP_APPSRV:buckie^Bapptest2:pts/0^B^B1180966869^B^BDS_DATABASE_SERVERS:dbcorphd^Bsvr032:pts/15^B^B1173891966^B^B

etc....etc...and then:
SPLIT=^B
FIELDS=USER TTY ROUTE TIME INFO
NRECORDS=669
$SERVCVER=6.0.3

With awk, line is too long...


So, in the above,

CDRP:fapirwan is the userid
svr607:pts/4 is the server and tty
1180963725 must be login time?

Should look like:

User Tty Time
CDRP:fapirwan pc0607:pts/4 06/04/07 07:28:45

Which is what I get from teh "web page" - but I want to do it from the command line instead...

Rgds...Geoff

I'm pretty sure perl is the answer (isn't it the answer to everything?)

I think this is one for my friend Hein.

Rgds...Geoff

[H.Merijn Brand (procura]

if

while (<>) {
my @row = split m/\cB/ => $_;
print "first field: ", $row[0], "\n";
}

doesn't work, you might like to have a look at Text::CSV_XS

use IO::Handle;
use Text::CSV_XS;
my $csv = Text::CSV_XS->new ({ sep_char => "\cB", binary => 1 });
while (my $row = $csv->getline (*ARGV) {
print "first field: ", $row->[0], "\n";
}

Enjoy, Have FUN! H.Merijn

[Geoff Wild]

Merijn,

Well - I tried:

#!/usr/bin/perl

foreach (`boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\* |cat -v`) {
chomp;
my @row = split m/\cB/ => $_;
print "first field: ", $row[0], "\n";
}


Just adds first field: to front of each line...doesn't seem to split on the ^B

Rgds...Geoff

[H.Merijn Brand (procura]

cat -v is munging you Ctrl-B to being not binary anymore :)

Then you're using foreach, which will read the whole dataset at once. And don't use backticks when you can do without.

open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*' or die "$!";
while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
print "first field: ", $row[0], "\n";
}

Enjoy, Have FUN! H.Merijn

[Geoff Wild]

Cool - getting closer...

#!/usr/bin/perl

$format="%20s %20s %20s\n";
printf $format, "Userid","Server:TTY","Time";


open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*' or die "$!";
while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
printf $format, $row[0], $row[1], $row[2], "\n";
}


Only get first row...how to get the rest?

Rgds....Geoff

[H.Merijn Brand (procura]

You should get all of the rows, unless the rows are not sperated by newlines.

The *field*seperator is a Ctrl-B, your snippet deals with the data as if the *record*separator is a \n (newline). If it is not, but e.g. it is a Ctrl-C, add

if (open my $dta, ...
local $/ = "\cC";
while (<$dta>) {
:
}
}

Enjoy, Have FUN! H.Merijn

[Geoff Wild]

That's correct - for some reason there is no new lines (strange output I know).

With the cat -v - it looks like each "line" is separated by 2 ^B's ???

Rgds...Geoff

[H.Merijn Brand (procura]

if there are no empty fields, you can set the line sep to "\cB\cB", and the rest will still work ok.

Enjoy, Have FUN! H.Merijn

[Geoff Wild]

You are getting me close - now just a format issue and I guess I need to convert the login time from for example "1180963725" to "06/04/07 07:28:45"

Here's the current script:

#!/usr/bin/perl

$format="%20s %20s %20s\n";
printf $format, "Userid","Server:TTY","Time";


if (open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!"){
local $/ = "\cB";

while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
printf $format, $row[0], $row[1], $row[2], "\n";
}
}


Part of output:

# ./z |more
Userid Server:TTY Time
DATA=CDRP:fapirwan
svr607:pts/4

1180963725

CDRP_PROD:cdrbatch
svr607:pts/5

1180965178

CISCOWORKS:kfulford
tmonprod01:pts/1

1180015709

CISCOWORKS:kfulford
tmonprod01:pts/2

1180038015


Rgds...Geoff

[Geoff Wild]

Interesting, this

printf $format, $row[0], $row[1], $row[2], "\n";

is now the same as

printf $format, $row[0];


IE - same output...

I guess the local $/ = "\cB"; is changing what is split....


Rgds...Geoff

[H.Merijn Brand (procura]

Why the | head -1?
And the last shot only showed a single \cB in $/

#!/usr/bin/perl

use strict;
use warnings;

my $format = "%20s %20s %20s\n";
printf $format, "Userid", "Server:TTY", "Time";

if (open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!") {
local $/ = "\cB\cB";

while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
my @tm = localtime $row[2]; # Assuming unix time stamps
printf $format, @row[0,1],
sprintf ("%2d-%02d-%4d %02d:%02d:%02d",
$tm[3], $tm[4] + 1, $tm[5] + 1900,
@tm[2,1,0]), "\n";
}
}

Enjoy, Have FUN! H.Merijn

[Geoff Wild]

the head -1 to get rid of the extra data fromt the command:

SPLIT=
FIELDS=USER TTY ROUTE TIME INFO
NRECORDS=688
$SERVCVER=6.0.3


Rgds...Geoff

[James R. Ferguson]

Hi Geoff:

> I guess I need to convert the login time from for example "1180963725" to "06/04/07 07:28:45"

That's easily added:

#!/usr/bin/perl
use POSIX qw (strftime);

printf "%-20s\n", strftime "%m/%d/%Y %T",localtime($time);


...where $time is the field representing your Epoch seconds.

Regards!

...JRF...

[Geoff Wild]

Okay - looking better - I get a bunch of:

Use of uninitialized value in localtime at ./zz line 15, <$dta> chunk 1036.
Use of uninitialized value in printf at ./zz line 16, <$dta> chunk 1036.


DATA=CDRP:fapirwan svr607:pts/4 31-12-1969 17:00:00
1180963725 31-12-1969 17:00:00
CDRP_PROD:cdrbatch svr607:pts/5 31-12-1969 17:00:00
1180965178 31-12-1969 17:00:00
CISCOWORKS:kfulford tmonprod01:pts/1 31-12-1969 17:00:00
1180015709 31-12-1969 17:00:00

[H.Merijn Brand (procura]

while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
my $s_time = "?";
if ($row[2]) {
my @tm = localtime $row[2];
$s_time = sprintf "%2d-%02d-%4d %02d:%02d:%02d", $tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];
}
printf $format, @row[0,1], $s_tim, "\n";
}
}

Enjoy, Have FUN! H.Merijn

[Geoff Wild]

Line 20 is:

printf $format, @row[0,1], $s_time, "\n";


Maybe, it's not row[2]?



Userid Server:TTY Time
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 2.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 4.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 6.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 8.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 10.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 12.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 14.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 16.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 18.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 20.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 22.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 24.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 26.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 28.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 30.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 32.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 34.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 36.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 38.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 40.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 42.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 44.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 46.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 48.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 50.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 52.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 54.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 56.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 58.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 60.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 62.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 64.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 66.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 68.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 70.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 72.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 74.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 76.
Use of uninitialized value in printf at ./zz line 20, <$dta> chunk 78.
DATA=CDRP:fapirwan svr607:pts/4 ?
1180963725 ?
CDRP_PROD:cdrbatch svr607:pts/5 ?
1180965178 ?
CISCOWORKS:kfulford tmonprod01:pts/1 ?
1180015709 ?

[Geoff Wild]

If I

print $row[0];

then I get one long output...

DATA=CDRP:fapirwansvr607:pts/41180963725CDRP_PROD:cdrbatchsvr607:pts/51180965178CISCOWORKS:kfulfordtmonprod01:pts/1118001570
9

If I:

print $row[0],"\n";

Userid Server:TTY Time
DATA=CDRP:fapirwan
svr607:pts/4

1180963725

CDRP_PROD:cdrbatch
svr607:pts/5

1180965178

CISCOWORKS:kfulford
tmonprod01:pts/1

1180015709



If I try
print $row[2];

I get nada....

[Geoff Wild]

Okay - just doing some troubleshooting...

If I:

#!/usr/bin/perl

$format="%20s %20s %20s\n";
printf $format, "Userid","Server:TTY","Time";


if (open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!"){
local $/ = "\cB\cB";

while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
printf $format, $row[0], $row[1];
}
}

Output is:

# ./z |more
Userid Server:TTY Time
DATA=CDRP:fapirwan svr607:pts/4
1180963725
CDRP_PROD:cdrbatch svr607:pts/5
1180965178
CISCOWORKS:kfulford tmonprod01:pts/1
1180015709


What that tells me is, DATA=CDRP:fapirwan is row[0] and svr607:pts/4 is row[1].


For some reason, the time is also row[0]



Looking at the orig data:

# boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|cat -v
DATA=CDRP:fapirwan^Bsvr607:pts/4^B^B1180963725^B^BCDRP_PROD:cdrbatch

we see 2 ^B's before AND after the time field :(

[Geoff Wild]

Here's where I am now:

#!/usr/bin/perl

use strict;
use warnings;

my $digits = '\d+';

my $format = "%20s %20s %20s\n";
printf $format, "Userid", "Server:TTY", "Time";

if (open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!") {
local $/ = "\cB\cB";

while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
my $s_time = "?";
if ($row[0] =~ $digits) {
my @tm = localtime $row[0];
$s_time = sprintf "%2d-%02d-%4d %02d:%02d:%02d", $tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];
}
else {
$s_time = " ";
}
printf $format, @row[0,1], $s_time, "\n";
}
}


I still get a lot of

Use of uninitialized value in print at ./zz line 25, <$dta> chunk 870.

And format not quite right:

DATA=CDRP:fapirwan svr607:pts/4
1180963725 4-06-2007 07:28:45
CDRP_PROD:cdrbatch svr607:pts/5
1180965178 4-06-2007 07:52:58
CISCOWORKS:kfulford tmonprod01:pts/1
1180015709 24-05-2007 08:08:29


It would be nice to somehow join the time field to be on same line as the first 2 fields...and not print out the "Unix" time at all - just the converted time...

Rgds...Geoff

[Hein van den Heuvel · Accepted solution]

That should be easy, but a little tricky with the actual data. [attach?]
It seems to me there is a new line coming from the $format, as well as from a print argument.


Here what works for me, after replacing th ^B^B in the sample line with ~ and ^B with and just 'typing' the dataline ;

C:\temp>perl y.pl
Userid Server:TTY Time
CDRP:fapirwan svr607:pts/4 4-06-2007 09:28:45
CDRP_PROD:cdrbatch svr607:pts/5 4-06-2007 09:52:58
CISCO:kfulford tmonprod01:pts/1 24-05-2007 10:08:29
CISCO:kfulford tmonprod01:pts/2 24-05-2007 16:20:15
CORP_APPSRV:buckie apptest2:pts/0 4-06-2007 10:21:09
DS_DATABASE_SERVERS:dbcorphd svr032:pts/15 14-03-2007 13:06:06

-------------- y.pl -------------------
#!/usr/bin/perl

use strict;
use warnings;
my $format = "%20s %20s %20s\n";
printf $format, "Userid", "Server:TTY", "Time";

if (open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!") {
local $/ = "\cB\cB";

while (<$dta>) {
chomp;
my @row = split m/\cB/ => $_;
my $s_time = "?";
if ($row[0] =~ $digits) {
my @tm = localtime $row[0];
$s_time = sprintf "%2d-%02d-%4d %02d:%02d:%02d", $tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];
}
else {
$s_time = " ";
}
printf $format, @row[0,1], $s_time, "\n";
}
}


--------------- real code ?? --------------
#!/usr/bin/perl

use strict;
use warnings;

my $format = "%20s %20s %20s\n";
printf $format, "Userid", "Server:TTY", "Time";

my (@row, $i);

open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!";
local $/ = "\cB\cB";

while (<$dta>) {
chomp;
s/^DATA=// unless $i++; # clean up header on first entry
if (/^(\d+)/) { # line with timestamp ?
my @tm = localtime $1;
my $s_time = sprintf "%2d-%02d-%4d %02d:%02d:%02d",
$tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];
printf $format, @row[0,1], $s_time;
} else {
@row = split m/\cB/;
}
}

Good luck!

[Hein van den Heuvel]

Oops, some cut & paste error.
This the script that worked on the sample data as I munged it:

DATA=CDRP:fapirwan;svr607:pts/4~1180963725~CDRP_PROD:cdrbatch;svr607:pts/5~1180965178~CISCO:kfulford;tmonprod01:pts/1~1180015709~CISCO:kfulford;tmonprod01:pts/2~1180038015~CORP_APPSRV:buckie;apptest2:pts/0~1180966869~DS_DATABASE_SERVERS:dbcorphd;svr032:pts/15~1173891966~


#!/usr/bin/perl

use strict;
use warnings;

my $digits = '\d+';

my $format = "%20s %20s %20s\n";
printf $format, "Userid", "Server:TTY", "Time";

my (@row, $i);

open my $dta, "-|", 'type x.txt' or die "$!";
local $/ = '~';
while (<$dta>) {
chomp;
s/^DATA=// unless $i++; # clean up header on first entry
if (/^(\d+)/) { # line with timestamp ?
my @tm = localtime $1;
my $s_time = sprintf "%2d-%02d-%4d %02d:%02d:%02d",
$tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];
printf $format, @row[0,1], $s_time;
} else {
@row = split m/;/;
}
}

[Geoff Wild]

Thanks to Merijn and Hein for this most excellent perling!

Here's my final script:

#!/usr/bin/perl
#
# users-logged-in.pl
#
#
# Perl script to get a list of users logged in from the command line
#
# Note: must be in boks shell

use strict;
use warnings;

my $format = "%35s %35s %30s\n";
printf $format, "Userid", "Server:TTY", "Date Time";

my (@row, $i);

open my $dta, "-|", 'boksauth -Oresults -c FUNC=read TAB=5 FIELDS=\*|head -1' or die "$!";
local $/ = "\cB\cB";

while (<$dta>) {
chomp;
s/^DATA=// unless $i++; # clean up header on first entry
if (/^(\d+)/) { # line with timestamp ?
my @tm = localtime $1;
my $s_time = sprintf "%2d-%02d-%4d %02d:%02d:%02d",
$tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];
printf $format, @row[0,1], $s_time;
} else {
@row = split m/\cB/;
}
}

[Hein van den Heuvel]

Hey Geoff.
Good to see it works!

The pipe to head is silly though!
Very non-perl thinking.

Much like the frequently seen:
cat file | grep text | awk '
Using 2 pipes instead of direct:
awk '/text/...' file

Just tell perl to do it!
Let it count lines or let it look for "\nSPLIT" (similar to the removing of "DATA=")

Now that I looked at the original topic again I have to warn you about the ^B^B line terminator approach.

>> SPLIT=^B
>> FIELDS=USER TTY ROUTE TIME INFO

It appears that the ^B^B is just an artifact of ROUTE and INFO being empty.
If those are ever filled, the script will need to be adapted.

If those new-lines after the lines are real (such as head suggest) then the data format has the potential to parse the data largely free format.

I realize this is excessive for the problem on hand but in the interested of learning and understanding consider the variant below.
The data really wants to be treated as a stream, but unfortunately the seperator is not known untill beyond the data stream. So you either have to read twice, or build an arbetrary long variable/array. Yuck to both.

Cheers,
Hein.


#!/usr/bin/perl
use strict;
use warnings;
my $format = "%20s ";

my (@fields, @values, $data, $split, $nrecords, $timefield, $i, $n, $text, $line);
#
# Get all data
#
open my $dta, "-|", 'type x.txt' or die "$!";
while (<$dta>) {
chomp;
$data = $1 if /^DATA=(.*)/;
$split = $1 if /^SPLIT=(.*)/;
@fields = split (/\s+/,$1) if /^FIELDS=(.*)/;
$nrecords = $1 if /^NRECORDS=(.*)/;
}
#
# Process header
#
@values = split $split, $data;
for ($i=0; $i<@fields; $i++) {
$timefield = $i if $fields[$i] eq 'TIME';
$text .= sprintf $format, $fields[$i];
$line .= sprintf $format, '-'x(length($format) - 1);
}
chop $text;
chop $line;
print "$text\n$line\n";
#
# Process data lines
#
$n=0;
while ($n<@values-@fields) {
$text = "";
for ($i=0; $i<@fields; $i++) {
#
# Special formatting for time field
#
if ($i==$timefield) {
my @tm = localtime $values[$n];
$values[$n] = sprintf "%2d-%02d-%4d %02d:%02d:%02d",
$tm[3], $tm[4] + 1, $tm[5] + 1900, @tm[2,1,0];

}
$text .= sprintf $format, $values[$n++];
}
print $text."\n";
}
#
# All done.
#
$n /= @fields;
print "\n\n$n records seen, $nrecords expected\n";