topic Re: Scripting help in Operating System - HP-UX

Scripting help

Sean OB_1 — Wed, 04 Aug 2004 11:23:22 GMT

Can anyone rewrite this script snippet so that it will be more efficient?

for PWDRECORD in `cat '/etc/passwd'`
do
LOG_ID=`echo $PWDRECORD|cut -d':' -f1`
COSMO_ID=`echo $PWDRECORD|cut -d',' -f4`
if [ "$LOG_ID" = "" -o "$COSMO_ID" = "" ]
then
continue
fi
if [ `echo "$COSMO_ID" | grep ":" > /dev/null` ]
then
continue
fi
STUDENT_EMPLOYEE=`echo $PWDRECORD|cut -d ',' -f5`
STUDENT_EMPLOYEE=`echo $STUDENT_EMPLOYEE | cut -c1`
( echo $COSMO_ID | grep $LOG_ID >/dev/null )|| ( [ ! "$COSMO_ID" = "0" ] && \
echo $COSMO_ID","$LOG_ID","$STUDENT_EMPLOYEE >> /tmp/bbids.list )
unset LOG_ID
unset COSMO_ID
unset STUDENT_EMPLOYEE
done

TIA and points for all responses.

Sean

Re: Scripting help

Dietmar Konermann — Wed, 04 Aug 2004 12:37:22 GMT

No idea if I got your logic right... :-) however, this should be *much* faster. A pure perl or awk solution could be a lot more fast, of course.

while IFS=: read LOG_ID x x x FIELD5 x
do
typeset -L1 STUDENT_EMPLOYEE
echo $FIELD5 | IFS=, read x x x COSMO_ID STUDENT_EMPLOYEE x
[[ -z "$LOG_ID" || -z "$COSMO_ID" || "$COSMO_ID" = *:* ]] && continue
[[ ! $COSMO_ID = *$LOG_ID* && ! "$COSMO_ID" = "0" ]] && echo "$COSMO_ID,$LOG_ID,$STUDENT_EMPLOYEE"
unset COSMO_ID
unset STUDENT_EMPLOYEE
unset STUDENT_EMPLOYEE
done < passwd

Best regards...
Dietmar.

Re: Scripting help

Kent Ostby — Wed, 04 Aug 2004 13:08:08 GMT

Sean -- can you give me a sample input and output lines.

If so, I can write an awk script that should be fairly clean for this.

Best regards,

Kent M. Ostby

Re: Scripting help

curt larson_1 — Wed, 04 Aug 2004 13:15:28 GMT

cat /etc/passwd |
awk -F":" '{
logId=$1;
l1=length(logId);
# if logId = "" do next record
if ( l1 == 0 ) next;
gecos=$5;
split(gecos,a,",");
cosmoId=a[4];
l1=length(cosmoId);
if ( l1 == 0 ) next;
if ( cosmoId ~ ":" ) next;
stdEmp=substr(a[5],1,1);
if ( cosmoId !~ logId && cosmoId != "0" )
print cosmoId,logId,stdEmp;
}' >/tmp/bbids.list

that is a good start on an awk script

Re: Scripting help

Sean OB_1 — Wed, 04 Aug 2004 13:17:26 GMT

Thanks, testing it now to see if it comes out with the same results as the original.

Can you take a minute to explain to me what this is doing? I mean your script and how it works.

Re: Scripting help

Sean OB_1 — Wed, 04 Aug 2004 13:20:07 GMT

Sorry that reply was meant for Dietmar.

Kent, here's sample input:

aaronk:x:43280:20:Aaron,Karema,D,0296002,S:/false/home:/bin/false

and output:

0296002,aaronk,S

Re: Scripting help

Sean OB_1 — Wed, 04 Aug 2004 13:25:53 GMT

Curt,

That's pretty close except that it is putting spaces instead of commas between the entries. I stuck in commas, but it still puts in the spaces. Is there a way to suppress the spaces?

Re: Scripting help

Sean OB_1 — Wed, 04 Aug 2004 13:29:07 GMT

Curt,

hmm, not quite right, even after I stripped out the spaces manually:

bbids.list is the orig, bbids.lst2 is from your script:

root@cosmo0:/tmp/sean-> diff bbids.list bbids.lst2
1c1
< 9123456,aaaaaaaa,S
---
> 9.12346e+06,aaaaaaaa,S
267c267
< 8675309,acme001,S
---
> 8.67531e+06,acme001,S
2869c2869
< EXM20031026023108.CSV:0924004,ashleyt,E
---
> EXM20031026023108.CSV,ashleyt,
20019c20019
< 8888888,doofusj,E
---
> 8.88889e+06,doofusj,E
68532c68532
< 9876543,ptest,E
---
> 9.87654e+06,ptest,E

Here are the original entries for those records.

doofusj:x:72495:20:Doofus,Joe,P,8888888,E:/home/doofusj:/usr/local/matc/bin/cosmo
ptest:x:99910:20:Fanning,Pete,W,9876543,E:/home/ptest:/usr/local/matc/bin/cosmo
ashleyt:x:38154:20:Ashley,Terry,,EXM20031026023108.CSV:0924004,E:/home/ashleyt:/usr/local/matc/bin/cosmo
acme001:x:50000:20:Neumann,Alfred,E,8675309,S:/false/home:/bin/false
aaaaaaaa:x:1909:20:TestLast,TestFirst,M,9123456,S:/false/home:/bin/false

Re: Scripting help

Kent Ostby — Wed, 04 Aug 2004 13:30:18 GMT

Sean -- This has less error checking then you're original script, but here you go:

awk '{FS=":"; u5=$5; p2=$1; $0=u5; FS=","; print $4","p2","$5;}' passwd

Essentially we are printing the fourth part of the fifth field then a comma then the first field then the fifth part of the fifth field.

Hope that helps.

Oz

Re: Scripting help

curt larson_1 — Wed, 04 Aug 2004 13:36:18 GMT

a couple of improvements

cat /etc/passwd |
awk -F":" '{
logId=$1;
# if logId = "" do next record
if ( ! logId ) next;
gecos=$5;
split(gecos,a,",");
cosmoId=a[4];
if ( ! cosmoId ) next;
stdEmp=substr(a[5],1,1);
if ( cosmoId !~ logId && cosmoId != "0" )
print cosmoId,logId,stdEmp;
}' >/tmp/bbids.list

Re: Scripting help

curt larson_1 — Wed, 04 Aug 2004 13:39:10 GMT

for the printing do
printf("%s,%s,%s\n",cosmoId,logId,stdEmp);
instead of
print cosmoId,logId,stdEmp;

Re: Scripting help

Hein van den Heuvel — Wed, 04 Aug 2004 13:43:20 GMT

I can not exactly follow the intention of the script, but the following awk code should be pretty darn close, or at least serve as 'food for thought'.

BEGIN {IFS=":";OFS=","}
{
LOG_ID = $1;
split($0,commas,",");
COSMO_ID=commas[3];
if (LOG_ID COSMO_ID == "") { next };
STUDENT_EMPLOYEE=substr(commas[4],1,1);
if (index(COSMO_ID,LOG_ID) || (COSMO_ID != 0)) {
print COSMO_ID, LOG_ID, STUDENT_EMPLOYEE >> "/tmp/bbid.lst";
}
}

fwiw,
Hein.

Re: Scripting help

Sean OB_1 — Wed, 04 Aug 2004 14:08:39 GMT

Thanks everyone. The results are in. The original ran for 5 hours this morning. The new script took 11 seconds. :-)

I knew I could count on the ITRC forum crew!