topic Re: Awk Programming help in Operating System - Linux

Awk Programming help

Geoff Wild — Wed, 03 May 2006 10:18:37 GMT

I can probably figure this out - but I have to generate a list in lees then 2 hours...

I have input from a command like so (for hundreds of users):

Username: DS_UNIX_SERVERS:gwild
User ID: 29125
User Class: DS_UNIX_ADMIN
Group ID: 4
Comment: Geoffrey Wild - DS Unix Admin
Home directory: /home/gwild
Shell: /bin/ksh
File permission mask (umask):
Program Search Path:
Start Program:
Inactivity timeout: 0 minutes
Time dependent timeout: no
Inactivity timeout checking: CPU, keyboard, screen
Forced (hard) one-time password: yes
Serial number of Password Generator: 1234
Type of password generator: SecurId
Password last changed: 03/24/03
Password valid until: 04/22/07
User valid until: 03/23/07
Last login: 05/01/06 15:16:42
Last logout: 05/02/06 14:20:26
Number of failed logins: 0
User blocked: no

I need to print it like:

Host Group Userid User Class Password Expires Password Generator
DS_UNIX_SERVERS gwild DS_UNIX_ADMIN 04/22/07 SecurId

Thanks...Geoff

Re: Awk Programming help

Ninad_1 — Wed, 03 May 2006 10:33:37 GMT

Geoff,

You need to do the following

cat filename | egrep -i 'username|user id|user class|password valid until|type of password generator' | \
awk -F: '{a=$2;getline;b=$2;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Regards,
Ninad

Re: Awk Programming help

Rodney Hills — Wed, 03 May 2006 10:35:10 GMT

Maybe something like-

awk -F: '/^Username/{host=$1;userid=$2};
/^User Class/{userclass=$1};
... repeat above for each data item you want
/^User blocked/{print host,userid,userclass,...}' inputfile

HTH

-- Rod Hills

Re: Awk Programming help

Ninad_1 — Wed, 03 May 2006 10:37:55 GMT

Geoff,
Sorry small correction

cat filename | egrep -i 'username|user id|user class|password valid until|type of password generator' | \
awk -F: '{a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Regards,

Ninad

Re: Awk Programming help

Ninad_1 — Wed, 03 May 2006 10:41:14 GMT

Geoff,
Sorry small correction

cat filename | egrep -i 'username|user class|password valid until|type of password generator' | \
awk -F: '{a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Regards,

Ninad

Re: Awk Programming help

Mike Stroyan — Wed, 03 May 2006 10:47:16 GMT

Here is a version that picks each data item out by the leading pattern. It just substitutes a space for the : in the "Host Group Userid" pair. It didn't seem worth parsing them out into two separate variables. The output is triggered by getting a "Password valid until:" line.
That presumes the fields are always in the same order. It will handle a file that contains the same pattern repeated for many users.

BEGIN{print "Host Group Userid User Class Password Expires Password Generator"}
/^Username:/{Username=$NF ;sub(":"," ",Username)}
/^User Class:/{UserClass=$NF}
/^Type of password generator:/{PasswordGenerator=$NF}
/^Password valid until:/{PasswordExpires=$NF;
print Username " " UserClass " " PasswordExpires " " PasswordGenerator}

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 10:55:44 GMT

Ninad - close...just need to format...

Rodney - havn't tried it yet...

Mike - yours looks interesting...

Just so you know, it isn't a filename - but output from a command...

Something like:

lsbks -a -l $i | awk...

where $i is DS_UNIX_SERVERS:gwild

Rgds...Geoff

Re: Awk Programming help

Steven E. Protter — Wed, 03 May 2006 11:00:07 GMT

Not on thread topic.

Having some trouble replying to your firewall question. Domain keeps coming up bad.

My domain.

Domain of sender address
Myemail@mydomain.com does not exist

I've sent to you before and my MX record looks up. Perhahps provide an alternate means of me reaching you.

Good luck with the awk, I'm too lazy today to work on it.

SEP

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 11:04:37 GMT

SEP - can you go to chat?

http://www.cmve.net/voc/voc.php

Re: Awk Programming help

Ninad_1 — Wed, 03 May 2006 11:07:49 GMT

Geoff,

This will give you what you want in the required format as well.

lsbks -a -l $i | egrep -i 'username|user class|password valid until|type of password generator' | \
awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Regards,
Ninad

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 11:17:34 GMT

Ninad - nope - format is not right...

# lsbks -a -l DS_UNIX_SERVERS:gwild | egrep -i 'username|user class|password valid until|type of password generator' |
awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'> awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'
Host Group Userid User Class Password Expires Password Generator
DS_UNIX_SERVERS gwild DS_UNIX_ADMIN 04/22/07 SecurId
Keon

I included a txt file so you can see the formatting...

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 11:20:19 GMT

Mike - yours seems to bail as well:

# lsbks -a -l DS_UNIX_SERVERS:gwild |awk 'BEGIN{print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}'

Host Group Userid User Class Password Expires Password Generator
DS_UNIX_SERVERS:gwild ID: ID: Class:
Geoffrey Wild directory: permission /bin/ksh
Search Path: Program: dependent timeout:
timeout checking: (hard) of number
last changed: valid login: valid
logout: 05/02/06 of blocked:

Re: Awk Programming help

Ninad_1 — Wed, 03 May 2006 11:26:06 GMT

Geoff,

OK, the tab had probably not come in the previous post. Just a bit of modification. add the tr at the end, so final command would be

lsbks -a -l $i | egrep -i 'username|user class|password valid until|type of password generator' | awk -F: 'BEGIN {print "Host Group Userid User Class Password Expires Password Generator"} {a=$2;b=$3;getline;c=$2;getline;e=$2;getline;d=$2;print a,b,c,d,e}' | tr -s " " " "

Note the tab between the first " quotes in tr and space in the second " quotes.

Regards,
Ninad

Re: Awk Programming help

Peter Nikitka — Wed, 03 May 2006 11:34:46 GMT

Hi,

give that a try:

... | awk 'BEGIN {print "Host ..."}
/^Username: / { for (i=1;iu=0
split($2,us,":"); dat[++u]=us[1]; dat[++u]=us[2]
}
/^User Class:/ {dat[++u]=$3}
...
MORE MATCHES TO FILL dat-array
...
END {for (i=1;i
mfG Peter

Re: Awk Programming help

Hein van den Heuvel — Wed, 03 May 2006 11:41:23 GMT

I don't like relying on hardcoded order.
Here is a solution which look for each line.

awk -F: '{gsub (/ |\t/,"",$0)} /name:/{h=$2;u=$3} /lass:/{c=$2} /tor:/{t=$2} /wordva/{v=$2} /failed/{print h,u,c,v,t}'

- split input on ":"
- replace all tabs and spaces with nothing
- look for unique strings to identify lines
- for each desired line remember the value in variable
- Print all variables at known last line
(I picked 'failed', could double-up by picking 'Passwordvalid')
- Set OFS as desired (comma? tab?)

Hein.

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 11:54:02 GMT

Looks like Peter's is the best - thanks all...

Rgds...Geoff

Re: Awk Programming help

Steven E. Protter — Wed, 03 May 2006 11:59:14 GMT

No joy on chat.

Here is my reply to your query.

I've considered doing it for a long time.

NDS did it on a project for me a few months ago to stop script kiddies
auto login attempts.

It worked nicely

If I had it to do in iptables, your way is how I'd do it.

iptables -D INPUT -m tcp -p tcp --dport 22 -j DROP
# were my default now dropping
iptables -A INPUT -m tcp -p tcp --dport 22 --source $IPRANGE1 -j ACCEPT
iptables -A INPUT -m tcp -p tcp --dport 22 --source $IPRANGE2 -j ACCEPT
iptables -A INPUT -m tcp -p tcp --dport 22 --source $IPRANGE3 -j ACCEPT #
we now have a range perhaps covering office and home.

iptables -A INPUT -m tcp -p tcp --dport 22 -j DROP
# Redundant, I think this last line should be dropped.

Make sure your range covers a fixed address where you always have access.
I have a Shell account at speakeasy, so I make sure that server is covered
by an accept range. Then no matter where I am, I access speakeasy and I'm
in. Problem came up when speakeasy changed the network server address and
didn't tell me. Naturally this happened when I was sitting in an airport
waiting for a flight and needed to diagnose a problem.

Another thing to think about:
FC-5 has clustering built in. Even on a single node server, you can start
services with clustering instead of chkconfig and you get a built in
service monitor.

Have fun.

Interesting thread here.

SEP

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 12:05:40 GMT

SEP - yes - I put my way into production - works great.

As far as your email - my maillog is huge!

I'm at work and can't remember you email addy :(

Chat doesn't work for you eh? too bad...

I'll check when I'm at home tonight...

Re: Awk Programming help

Geoff Wild — Wed, 03 May 2006 12:15:21 GMT

As far as this thread - I'm thinking I'm using the wrong tool - should be perl.....

What I have now:

#!/bin/sh

#print a header
printf "Host Group \tUserid\tUser Class\tType\tPW Valid\tBlocked"

#get a list of id's
for i in `/opt/boksm/sbin/lsbks -a -l *:* -Dl`
do

lsbks -a -l $i | \
awk 'BEGIN {}
/^Username: / { for (i=1;i u=0
split($2,us,":"); dat[++u]=us[1]; dat[++u]=us[2]
}
/^User Class:/ {dat[++u]=$3}
/^Type of password generator:/ {dat[++u]=$5}
/^Password valid until:/ {dat[++u]=$4}
/^User blocked:/ {dat[++u]=$3}
END {for (i=1;i
done

Output is still kind of ugly, for is the user isn't on Securid, nothing for that field..

Sample output in attachment...

Re: Awk Programming help

Hein van den Heuvel — Wed, 03 May 2006 13:05:34 GMT

Here is a 'cute' perl approach.

All the words before the first colon become key to a data array, the value being the second field.

An execption is made for 'username', having a double value.

When the 'last' key is seen we
- deal with non-existing values as needed
- printf using the same format used to print the header, and just naming whatever fields desired
- clear the data array for the next round

------------ pretty.pl -----

$format="%20s %20s %20s %10s %10s %10s\n";
printf $format, "Host Group","Userid","User Class","PW Valid", "Type", "Blocked";

while (<>) {
s/\s+//g;
($key,$value,$extra)=split /:/;
$a{$key}=$value;
$a{name}=$extra if /^Username/;
if (/blocked/) {
$a{Typeofpasswordgenerator} = "NA" unless $a{Typeofpasswordgenerator};
printf $format, $a{Username}, $a{name}, $a{UserClass},
$a{Passwordvaliduntil}, $a{Typeofpasswordgenerator}, $a{Userblocked};
undef (%a);
}
}