Awk Programming help

[Geoff Wild]

I can probably figure this out - but I have to generate a list in lees then 2 hours...

I have input from a command like so (for hundreds of users):

Username: DS_UNIX_SERVERS:gwild
User ID: 29125
User Class: DS_UNIX_ADMIN
Group ID: 4
Comment: Geoffrey Wild - DS Unix Admin
Home directory: /home/gwild
Shell: /bin/ksh
File permission mask (umask):
Program Search Path:
Start Program:
Inactivity timeout: 0 minutes
Time dependent timeout: no
Inactivity timeout checking: CPU, keyboard, screen
Forced (hard) one-time password: yes
Serial number of Password Generator: 1234
Type of password generator: SecurId
Password last changed: 03/24/03
Password valid until: 04/22/07
User valid until: 03/23/07
Last login: 05/01/06 15:16:42
Last logout: 05/02/06 14:20:26
Number of failed logins: 0
User blocked: no


I need to print it like:

Host Group Userid User Class Password Expires Password Generator
DS_UNIX_SERVERS gwild DS_UNIX_ADMIN 04/22/07 SecurId


Thanks...Geoff

[Ninad_1]

Geoff,

You need to do the following

cat filename | egrep -i 'username|user id|user class|password valid until|type of password generator' | \
awk -F: '{a=$2;getline;b=$2;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Regards,
Ninad

[Rodney Hills]

Maybe something like-

awk -F: '/^Username/{host=$1;userid=$2};
/^User Class/{userclass=$1};
... repeat above for each data item you want
/^User blocked/{print host,userid,userclass,...}' inputfile

HTH

-- Rod Hills

[Ninad_1]

Geoff,
Sorry small correction

cat filename | egrep -i 'username|user id|user class|password valid until|type of password generator' | \
awk -F: '{a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'


Regards,

Ninad

[Ninad_1]

Geoff,
Sorry small correction

cat filename | egrep -i 'username|user class|password valid until|type of password generator' | \
awk -F: '{a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'


Regards,

Ninad

[Mike Stroyan]

Here is a version that picks each data item out by the leading pattern. It just substitutes a space for the : in the "Host Group Userid" pair. It didn't seem worth parsing them out into two separate variables. The output is triggered by getting a "Password valid until:" line.
That presumes the fields are always in the same order. It will handle a file that contains the same pattern repeated for many users.

BEGIN{print "Host Group Userid User Class Password Expires Password Generator"}
/^Username:/{Username=$NF ;sub(":"," ",Username)}
/^User Class:/{UserClass=$NF}
/^Type of password generator:/{PasswordGenerator=$NF}
/^Password valid until:/{PasswordExpires=$NF;
print Username " " UserClass " " PasswordExpires " " PasswordGenerator}

[Geoff Wild]

Ninad - close...just need to format...

Rodney - havn't tried it yet...

Mike - yours looks interesting...

Just so you know, it isn't a filename - but output from a command...

Something like:

lsbks -a -l $i | awk...

where $i is DS_UNIX_SERVERS:gwild

Rgds...Geoff

[Steven E. Protter]

Not on thread topic.

Having some trouble replying to your firewall question. Domain keeps coming up bad.

My domain.

Domain of sender address
Myemail@mydomain.com does not exist

I've sent to you before and my MX record looks up. Perhahps provide an alternate means of me reaching you.

Good luck with the awk, I'm too lazy today to work on it.

SEP

[Geoff Wild]

SEP - can you go to chat?

http://www.cmve.net/voc/voc.php

[Ninad_1]

Geoff,

This will give you what you want in the required format as well.

lsbks -a -l $i | egrep -i 'username|user class|password valid until|type of password generator' | \
awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'


Regards,
Ninad

[Geoff Wild]

Ninad - nope - format is not right...

# lsbks -a -l DS_UNIX_SERVERS:gwild | egrep -i 'username|user class|password valid until|type of password generator' |
awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'> awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'
Host Group Userid User Class Password Expires Password Generator
DS_UNIX_SERVERS gwild DS_UNIX_ADMIN 04/22/07 SecurId
Keon


I included a txt file so you can see the formatting...

[Geoff Wild]

Mike - yours seems to bail as well:

# lsbks -a -l DS_UNIX_SERVERS:gwild |awk 'BEGIN{print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Host Group Userid User Class Password Expires Password Generator
DS_UNIX_SERVERS:gwild ID: ID: Class:
Geoffrey Wild directory: permission /bin/ksh
Search Path: Program: dependent timeout:
timeout checking: (hard) of number
last changed: valid login: valid
logout: 05/02/06 of blocked:

[Ninad_1]

Geoff,

OK, the tab had probably not come in the previous post. Just a bit of modification. add the tr at the end, so final command would be

lsbks -a -l $i | egrep -i 'username|user class|password valid until|type of password generator' | awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}' | tr -s " " " "

Note the tab between the first " quotes in tr and space in the second " quotes.

Regards,
Ninad

[Peter Nikitka]

Hi,

give that a try:

... | awk 'BEGIN {print "Host ..."}
/^Username: / { for (i=1;iu=0
split($2,us,":"); dat[++u]=us[1]; dat[++u]=us[2]
}
/^User Class:/ {dat[++u]=$3}
...
MORE MATCHES TO FILL dat-array
...
END {for (i=1;i
mfG Peter

[Hein van den Heuvel]

I don't like relying on hardcoded order.
Here is a solution which look for each line.

awk -F: '{gsub (/ |\t/,"",$0)} /name:/{h=$2;u=$3} /lass:/{c=$2} /tor:/{t=$2} /wordva/{v=$2} /failed/{print h,u,c,v,t}'

- split input on ":"
- replace all tabs and spaces with nothing
- look for unique strings to identify lines
- for each desired line remember the value in variable
- Print all variables at known last line
(I picked 'failed', could double-up by picking 'Passwordvalid')
- Set OFS as desired (comma? tab?)

Hein.

[Geoff Wild]

Looks like Peter's is the best - thanks all...

Rgds...Geoff

[Steven E. Protter]

No joy on chat.

Here is my reply to your query.

I've considered doing it for a long time.

NDS did it on a project for me a few months ago to stop script kiddies
auto login attempts.

It worked nicely

If I had it to do in iptables, your way is how I'd do it.

iptables -D INPUT -m tcp -p tcp --dport 22 -j DROP
# were my default now dropping
iptables -A INPUT -m tcp -p tcp --dport 22 --source $IPRANGE1 -j ACCEPT
iptables -A INPUT -m tcp -p tcp --dport 22 --source $IPRANGE2 -j ACCEPT
iptables -A INPUT -m tcp -p tcp --dport 22 --source $IPRANGE3 -j ACCEPT #
we now have a range perhaps covering office and home.

iptables -A INPUT -m tcp -p tcp --dport 22 -j DROP
# Redundant, I think this last line should be dropped.

Make sure your range covers a fixed address where you always have access.
I have a Shell account at speakeasy, so I make sure that server is covered
by an accept range. Then no matter where I am, I access speakeasy and I'm
in. Problem came up when speakeasy changed the network server address and
didn't tell me. Naturally this happened when I was sitting in an airport
waiting for a flight and needed to diagnose a problem.

Another thing to think about:
FC-5 has clustering built in. Even on a single node server, you can start
services with clustering instead of chkconfig and you get a built in
service monitor.

Have fun.

Interesting thread here.

SEP

[Geoff Wild]

SEP - yes - I put my way into production - works great.

As far as your email - my maillog is huge!

I'm at work and can't remember you email addy :(

Chat doesn't work for you eh? too bad...

I'll check when I'm at home tonight...

[Geoff Wild]

As far as this thread - I'm thinking I'm using the wrong tool - should be perl.....

What I have now:

#!/bin/sh

#print a header
printf "Host Group \tUserid\tUser Class\tType\tPW Valid\tBlocked"

#get a list of id's
for i in `/opt/boksm/sbin/lsbks -a -l *:* -Dl`
do

lsbks -a -l $i | \
awk 'BEGIN {}
/^Username: / { for (i=1;i u=0
split($2,us,":"); dat[++u]=us[1]; dat[++u]=us[2]
}
/^User Class:/ {dat[++u]=$3}
/^Type of password generator:/ {dat[++u]=$5}
/^Password valid until:/ {dat[++u]=$4}
/^User blocked:/ {dat[++u]=$3}
END {for (i=1;i
done



Output is still kind of ugly, for is the user isn't on Securid, nothing for that field..

Sample output in attachment...

[Hein van den Heuvel]

Here is a 'cute' perl approach.

All the words before the first colon become key to a data array, the value being the second field.

An execption is made for 'username', having a double value.

When the 'last' key is seen we
- deal with non-existing values as needed
- printf using the same format used to print the header, and just naming whatever fields desired
- clear the data array for the next round

------------ pretty.pl -----

$format="%20s %20s %20s %10s %10s %10s\n";
printf $format, "Host Group","Userid","User Class","PW Valid", "Type", "Blocked";

while (<>) {
s/\s+//g;
($key,$value,$extra)=split /:/;
$a{$key}=$value;
$a{name}=$extra if /^Username/;
if (/blocked/) {
$a{Typeofpasswordgenerator} = "NA" unless $a{Typeofpasswordgenerator};
printf $format, $a{Username}, $a{name}, $a{UserClass},
$a{Passwordvaliduntil}, $a{Typeofpasswordgenerator}, $a{Userblocked};
undef (%a);
}
}

[James R. Ferguson]

Hi Geoff:

Here's a simple perl script (attached) that should meet your needs. You can run as:

# geoff.pl filename

(or):

# ... | geoff.pl

Regards!

[Geoff Wild]

Hein - nice format.

Just need to figure out how to pass data to it correctly - or maybe from within the script?

I did this:

for i in `/opt/boksm/sbin/lsbks -a -l *:* -Dl`
do
lsbks -a -l $i | /home/gwild/pretty.pl
done

Only thing is, it prints the "header" for every user...

Rgds...Geoff

[Hein van den Heuvel · Accepted solution]

Oh... I though you had one long stream of data coming anyway.

Make perl executed the listing, and act for each user.

Drop the suggested special end match, and just wait for the end of user data.

Untested:

-------- user_report.pl --------

#
# Heading
#
$format="%20s %20s %20s %10s %10s %10s\n";
printf $format, "Host Group","Userid","User Class","PW Valid", "Type", "Blocked";
#
# Find users
#
foreach (`/opt/boksm/sbin/lsbks -a -l *:* -Dl`) {
chomp;
$i = $_;
#
# Gather per user data
#
foreach (`/opt/boksm/sbin/lsbks -a -l $i`) {

s/\s+//g;
($key,$value,$extra)=split /:/;
$a{$key}=$value;
$a{name}=$extra if /^Username/;
}

#
# Tocuh up, print and clear
#
$a{Typeofpasswordgenerator} = "NA" unless $a{Typeofpasswordgenerator};

printf $format, $a{Username}, $a{name}, $a{UserClass},
$a{Passwordvaliduntil}, $a{Typeofpasswordgenerator}, $a{Userblocked};
undef (%a);

}

[Hein van den Heuvel]

>> I did this:
>
> for i in `/opt/boksm/sbin/lsbks -a -l *:* -Dl`
> do
> lsbks -a -l $i | /home/gwild/pretty.pl
> done

Not knowing lsbks I suspect you could just do:

lsbks -a -l *:* | pretty.pl


Also... Instead of the 'undef (%a)' at the bottom of a loop, I would normally write 'my %a;' at the top. But with my first script I assumend it being fed with a stream of data. So there is no top of the loop, and the end needs to be recognized anyway for printing purposes.

[back to real work]

Hein.

[Geoff Wild]

Hein's user_report.pl was the best.

Thanks all for the help.

Rgds...Geoff