topic Re: ksh script Q about removing entries from a list in a VAR in Operating System - Linux

ksh script Q about removing entries from a list in a VAR

Steve Post — Wed, 30 Jan 2008 13:55:17 GMT

I can think of how do this in perl, but I know it should be easily done in ksh.

I have a list of files in a variable.
MYLIST=`ls -1tr logs.*`
(That's one-t-r, not EL-t-r).
Because I ordered the files by time, the last two files on the list are the latest 2 files.

I want to remove the latest two logs from the list.

I can't wrap my brain around how to do this. Then I realized this is a good question for the forums. It would be good to see different methods to handle it.

In perl it would be:
( @goodlist, $last2, $last1) = ( @filelist );
But this isn't perl.

The script would be like this:
MYLIST=`ls -1tr logs.*`
# unknown magic code to remove latest 2 files goes here
for F in $MYLIST
do
rm $F
done

any help would be appreciated.

steve

Re: ksh script Q about removing entries from a list in a VAR

James R. Ferguson — Wed, 30 Jan 2008 14:00:38 GMT

Hi Steve:

# MYLIST=$(ls -1tr /tmp|tail -2|xargs)

Regards!

...JRF...

Re: ksh script Q about removing entries from a list in a VAR

James R. Ferguson — Wed, 30 Jan 2008 14:02:42 GMT

Hi (again) Steve:

Actually, since you just want to compose the list in the variable, you don't need the 'xargs'.

# MYLIST=$(ls -1tr /tmp|tail -2)

Regards!

...JRF...

Re: ksh script Q about removing entries from a list in a VAR

Steve Post — Wed, 30 Jan 2008 14:37:32 GMT

When I tried the answer I see I got the opposite of what I was looking for. It got the two files I want to take off the list. But since they are identified, I have a way to mask them out. Here is my code.

LAST2LIST=`ls -1tr log.?????? | tail -2`
LIST=`ls -1tr log.??????`
echo "orig"
echo "$LIST"
NLIST=
for Y in $LIST
do
Flag=0
for X in $LAST2LIST
do
if [ "$X" = "$Y" ] ; then
Flag=1
fi
done
if [ $Flag -eq 0 ] ; then
NLIST="$NLIST $Y"
fi
done
LIST="$NLIST"
echo "after"
for Y in $LIST
do
echo $Y
done

Perhaps there is a more elegant way to perform this?

Re: ksh script Q about removing entries from a list in a VAR

Eric SAUBIGNAC — Wed, 30 Jan 2008 15:04:14 GMT

Bonjour Steve,

Do you really need the list in a reverse order ? I mean do you use -r option only as a way to isolate the 2 latest files ?

If so, don't use -r option and have the latest files first. So it becomes simplier to remove them :

MYLIST=`ls -1t logs.* | tail -n +2`

Regards

Eric

Re: ksh script Q about removing entries from a list in a VAR

James R. Ferguson — Wed, 30 Jan 2008 15:07:21 GMT

Hi Steve:

I'm sorry, I didn't read your requirement very well!

NO POINTS PLEASE!

Anyway, one way in a shell would be to pipe the 'ls' output to a temporary file. Now, knowing the number of records in the file, loop, reading all but the last two.

By the way, in Perl your example would be:

# perl -le '@list=qw(a b c d e f);($x,$y,@good)=(@list);print "@good"'

In your post, '@goodlist' would greedily suck up all of '@filelist'.

Regards!

...JRF...

Re: ksh script Q about removing entries from a list in a VAR

Eric SAUBIGNAC — Wed, 30 Jan 2008 15:13:10 GMT

Sorry, did a mistake ;-(

It was :

MYLIST=`ls -1t logs.* | tail -n +3`

Eric

Re: ksh script Q about removing entries from a list in a VAR

Steve Post — Wed, 30 Jan 2008 15:36:25 GMT

Yep. I didn't really try the perl. If I did, I would have caught that.

Now the "tail -n -2" only grabs the last 2 lines. These are the two lines I don't want. I kept trying this and could not get it to work.

BUT.....I found an elegant way after all.

ls -1tr logs.* | sed '$d' | sed '$d'
This gives me the list of logs from earliest to latest. Then sed '$d' removes the bottom line. The second sed '$d' removes the bottom line again (ie the 2nd to last file).

Now perhaps there is something stupid about this I don't see. Maybe there is a sed command to remove the bottom two lines in one command instead of piping it twice?

steve

Re: ksh script Q about removing entries from a list in a VAR

James R. Ferguson — Wed, 30 Jan 2008 15:47:37 GMT

Hi Steve:

Eric has the simple solution. See _his_ _second _post_. It's the difference between +n and -n in 'tail' that I always forget!

NO POINTS PLEASE

Regards!

...JRF...

Re: ksh script Q about removing entries from a list in a VAR

Sandman! — Wed, 30 Jan 2008 15:49:08 GMT

# ls -1tr | awk '{x[NR]=$0} END {for(i=1;i

Re: ksh script Q about removing entries from a list in a VAR

Eric SAUBIGNAC — Wed, 30 Jan 2008 15:57:35 GMT

I find Sandman! post really attractive as it gives the answer with the reverse order that Steve wanted ;-)

Of course NO POINT

Eric

Re: ksh script Q about removing entries from a list in a VAR

Steve Post — Wed, 30 Jan 2008 16:59:27 GMT

Ok let me summarize.....
one way: ls -1tr | sed '$d' | sed '$d'
I don't like that I pipe to sed twice.

one way: ls -1t | tail -n +3
The +3 means to tail line 3 and up. (tail -n -3 would be the last 3 lines).
If the number of lines in the ls command is a very large number, the tail would fail. But this will be a small number of files.

one way: ls -1tr | awk '{x[NR]=$0} END {for(i=1;iThis awk script is nice. But I wouldn't use it until I knew people looking at it would understand it. And I would hope it worked on all versions of awk.

For me, I like the tail idea the best. But I REALLY like the fact that there is more than one way to do it.

Thank you all.

Re: ksh script Q about removing entries from a list in a VAR

Eric SAUBIGNAC — Wed, 30 Jan 2008 17:11:59 GMT

You said "But I wouldn't use it until I knew people looking at it would understand it. ". So I make the post for Sandman! as he can't control himself !

{x[NR]=$0}

==> For each line initialize an array (x) with the line itself ($0). NR is the current line number

END {for(i=1;i
==> at end of input stream, that is ls -1tr, just print all lines but the two last

It will work ... assuming that number of files listed is not greater that the limit of awk on array sizes.

No needs to give POINT

Regards

Eric

Re: ksh script Q about removing entries from a list in a VAR

James R. Ferguson — Wed, 30 Jan 2008 17:20:51 GMT

hi Steve:

So, if 'tail', sed' and 'awk' are OK for the filters, why not Perl?

# ls -1tr|perl -nle 'push @a,$_;END{print "@a[2..$#a]"}'

After all, TMTOWTDI.

Regards!

...JRF..