cancel
Showing results for 
Search instead for 
Did you mean: 

AWK assistance! Please

Chris Tijerina_2
Occasional Advisor

AWK assistance! Please

Yesterday I asked for some awk assistance and bascially if there was another way to untar a bunch of files and I did get some awesome replies. But, yet another question (
second part of the script) that I am stuck on, I have read manuals and I guess they are just to vague for me. PROBLEM: I am trying to untar a ton of baby tar balls that are identified (named) by their node name (we have over 500 nodes). I figured awk was the best way to do this, I did a 'ls -l' on the dir that the mother tar ball unloaded the baby tar balls into. The mother tar ball (1st part of script) created baby tar balls into another dir. I output the 'ls -l' to a file called 'babytar' and I awk the 'babytar' file and print on the 9th column which gives me the node name that I need to identify the baby tar balls by. I am getting an error on 'while' statement and the 'do' statement, if there is anybody that can spot what I am missing, again, I will CROWN THEE!
--------------Cut and Paste of Script----------
#!/bin/ksh

cd /home/s1perf/test_dir/test_dump
ls -l > /home/s1perf/test_dir/babytar
awk '{print $9}' /home/s1perf/test_dir/babytar | sort | while read server ; do
if [[ -f $DATA/$Server.tar ]] ; then
yes| tar xvf $Server.tar
fi
done
chmod 777 *

exit 0
"If you choose not to decide, you still have made a choice."
10 REPLIES
harry d brown jr
Honored Contributor

Re: AWK assistance! Please

upcase the "S" in server for the while ... do construct:

while read Server ; do

or change your other references of $Server with $server
Live Free or Die
Sridhar Bhaskarla
Honored Contributor

Re: AWK assistance! Please

You are good at giving awsome points too.. Thanks.

Now coming to your while statement, you are declaring the $ variable as "server" in the while loop but you are using "Server" in the subsequent lines. Case does matter in UNIX. So either replace "server" with Server or vice versa.

And it is a good practice to keep the $ variables in {}s if they are followed by extra characters. Like ${Server}.tar. It doesnt' matter much here but it does in some places.

-Sri
You may be disappointed if you fail, but you are doomed if you don't try
Sridhar Bhaskarla
Honored Contributor

Re: AWK assistance! Please

BTW.. I like your affinity towards "awk".

-Sri
You may be disappointed if you fail, but you are doomed if you don't try
Roger Baptiste
Honored Contributor

Re: AWK assistance! Please

Chris,

There seems to be a mismatch in the ""Server.tar""
and "'server"' in your script. Also, make sure
the DATA variable is defined.

-raj
Take it easy.
Curtis Larson_1
Valued Contributor

Re: AWK assistance! Please

here are some comments:

cd /home/s1perf/test_dir/test_dump
ls -l > /home/s1perf/test_dir/babytar
awk '{print $9}' /home/s1perf/test_dir/babytar

change to the above two lines to just:
ls -d | sort ...

if you really want the results of the "ls" in a file, do: ls -d | tee yourfilename | sort ...

| sort | while read server ; do
if [[ -f $DATA/$Server.tar ]] ; then

change $Server.tar to ${server}.tar
the {} isn't really necessary, but the S->s is
$DATA??? where is that defined?

yes| tar xvf $Server.tar

change $Server.tar to ${server}.tar
the {} isn't really necessary, but the S->s is

fi
done
chmod 777 *

if you want to change permissions on all the files you just untared, you'll need to add the -R option to the chmod if there are subdirectories
Robin Wakefield
Honored Contributor

Re: AWK assistance! Please

Hi Chris,

I assume the variable DATA is defined elsewhere, in which case the following should work for you:

========================================
#!/bin/ksh

cd /home/s1perf/test_dir/test_dump
ls | while read server ; do
test -f $DATA/${server}.tar && yes | tar xvf $DATA/${server}.tar
done
chmod 777 *

exit 0
=========================================

Don't forget that 'ls' sorts by default, and even though it outputs to the screen in multi-column format, internally it is one-per-line, so 'ls' on its own is OK.

I've added $DATA in the tar command too, don't know if that's what you want.

Rgds, Robin.
Volker Borowski
Honored Contributor

Re: AWK assistance! Please

Well, this seems to be quite complex (lots of pipes) ....

"ls -l | awk ..." vs. "ls -1" (one)
"while read" vs. "for name in"

This could be a streamlined version

#!/bin/ksh
cd /home/s1perf/test_dir/test_dump
for name in `ls -1` # add required sortflag to ls
do
if [[ -f $DATA/$name.tar ]] ; then
yes| tar xvf $name.tar
fi
done

The syntax error as already mentioned seems to me the uppercase "S" mismatch in your script.

Hope this helps
Volker
Carlos Fernandez Riera
Honored Contributor

Re: AWK assistance! Please


Volker ( and all):


`ls -1` create a new process, and can be changed by * that will be treated on the current shell.

Sort is not needed because you will treat whole directory, and * will show alfa. sorted.

I dont know why that yes.

So my command will be
for file in */*.tar
do
tar xvf $file
done

or

echo */*.tar | xargs -n 1 tar xvf

unsupported
Santosh Nair_1
Honored Contributor

Re: AWK assistance! Please

Just out of curiosity, why can't we replace the "ls |while read" construct with "for FILE in *" construct. The second construct saves on a process and a pipe but does the same thing...i.e.:


cd /home/s1perf/test_dir/test_dump
for server in *
do
test -f $DATA/${server}.tar && yes | tar xvf $DATA/${server}.tar
done
chmod 777 *
exit 0

-Santosh
Life is what's happening while you're busy making other plans
Volker Borowski
Honored Contributor

Re: AWK assistance! Please

Hi folks,
interesting thread !

Just did some tests on a UX 11 with root and other users in sh and csh.

Now did you expect, that

ls -1
is diffrent from
ls -1 *

???

OK, no suprise:
ls -1 * goes into directories and lists what's in in addition. This can be caught by "ls -1d".
BUT, suprise:
as root in sh, 'ls -1' lists hidden files while 'ls -1d *' does not.

Which is diffrent for a common user in csh.

Hey Chris, however you solve it, I recommend to verify that whatever wildcard you choose, it meets what you want.
:-)

Volker