How do I create a script to Purge all files in a Directory except for the latest 100???

[James R. Ferguson]

Shaun:

Please define what you mean by "latest". Is this a modification timestamp, or simply the last 100 files in a directory, as ordered by normal collation?

...JRF...

[Shaun Aldrich]

Thanks for the reply. To clarify this better I am looking for more of a script which will delete all but the latest 100 files in the directory according to timestamp.

I am not very good at all at creating scripts so any help is greatly appreciated!!!

Thanks, Shaun Aldrich

You can email me at SAldrich@chaptersinc.com

[Tom Danzig]

Try this:

rm `ls -1t | tail +102`

Please test this on some unneeded files first to insure it works as desired!

[Victor BERRIDGE]

rm $(ls -t|head -n100 )

[James R. Ferguson]

Shaun:

Try something like this:

FILES=`ls -t $MYDIR` #...collect by mod time
COUNT=`echo $FILES | wc -w` #...get how many
let N=COUNT-100 #...keep 100 of them...
FILES=`echo $FILES | cut -f1-$N -d " "`
rm $R

...JRF...

[James R. Ferguson]

Shaun:

Sorry, the script should look like:

Shaun:

Try something like this:

FILES=`ls -t $MYDIR` #...collect by mod time
COUNT=`echo $FILES | wc -w` #...get how many
let N=COUNT-100 #...keep 100 of them...
FILES=`echo $FILES | cut -f1-$N -d " "` #...list's head
rm $FILES

[Shaun Aldrich]

Thanks a lot James...

Is that the only 5 lines I would need in my script? Like I said, unfortunately my scripting ability is very poor right now.

Shaun Aldrich

[Victor BERRIDGE]

My first solution was too short?
OK
What about with FOR :
#
for FILE in `ls -t|head -n10`
do
rm $FILE
done

Regards
Victor

[James R. Ferguson]

Shaun:

Other than defining $MYDIR to that of your choosing, yes, the script is complete as written. You could define MYDIR as, for instance MYDIR=/tmp/log* to match all files beginning with "log" in the /tmp.

My suggestion is but one solution. Welcome to UNIX where there is always more than one way to do anything!!!

...JRF...

[Shaun Aldrich]

Hi Victor,

Thanks for the response. In the first line of the FOR LOOP what is the second word FILE specified for? Is it a variable?

Will this delete all of the files in the directory except for the latest 100 in terms of timestamp?

Your help is greatly appreciated...

[Victor BERRIDGE]

Sorry Shaun,
I was on the phone while writing the first post and got confused with what I was asked by phone...
Of course not my script would have erased the last 100..
Since you want to keep the last 100 there is the solution that James put down for you:
1) count first to know how many files to be removed
2) remove them.

Other solutions exists but James has an elegant one...

Best regards
Victor

[Victor BERRIDGE]

lets hope this one is a good one, so that you can forgive me...

for FILE in `ls |grep -v "$(ls -t|head -n100)"`
do
echo $FILE
done

James, I want your comments, what do you think of this?

Best regards
Victor

[Ed Hon]

What happens if there are less than 100 files?

[Victor BERRIDGE]

with the echo instead of rm I tried when ther is less and got not candidates echoed, thus me asking your advice (well James...) on can you trust this result?

Victor

[Shaun Aldrich]

Hi Victor,

Where do you put the rm command within the four lines of the statement you just sent me? All files but the latest 100 must be deleted in the directory. Does the rm command go in after that 4 line statement?

Thanks for your help Victor and others....

[Victor BERRIDGE]

Instead of :
echo $FILE

You should put rm $FILE

Best regards
Victor

[James R. Ferguson]

Shaun:

Victor's solution is an elegant solution too.

Shaun, you might want to add some logic to exit if there are less than 100 files. In total, a recap of my solution is as follows. Variables are in UPPERCASE.

MYDIR=/tmp/mylog* #...whatever...
FILES=`ls -t $MYDIR` #...collect by mod time
COUNT=`echo $FILES | wc -w` #...get how many
if (( COUNT < 100 ))
then
exit 0
fi
let N=COUNT-100 #...keep 100 of them...
FILES=`echo $FILES | cut -f1-$N -d " "` #...list's head
rm $FILES

...JRF...

[Tom Danzig]

rm `ls -1t | tail +101`

will work even if there are less than 100 files.

[James R. Ferguson · Accepted solution]

Shaun:

Please remember that points for those of us who have provided input are greatly appreciated!

...JRF...

[Shaun Aldrich]

Thanks a lot James/Victor...

Just a quick question for you James. In the last response you answered I noticed that in the following line -

FILES=`echo $FILES | cut -f1-$N -d " "` #...list's

in the cut portion you have -d. What does that do? If you get a chance what exactly does this whole line do?

I really like the logic that you put into this. I will have to learn this a lot better.

One other question is why do you have the head statement by itself above the rm?

Does it not need any options? Look forward to hear from you. Thanks

[Victor BERRIDGE]

Hi Tom,
Your solution is a nice one liner..
For our friend why not try to fit it in a script using while since our friend wants to learn the art of scripting

Regards
Victor

[CHRIS_ANORUO]

Check this script

cd (To directory)
ls -lrt|head -100 > ttt
ms=`ls -l|cat ttt|cut -c59-90`
set -hk $ms; nn=`echo $#`; n=1
while [ $n -le 2 ]
do
me=`ls -l|cat ttt|cut -c59-90`
rm -i $me
n=`expr $n+1`
shift 1
done

[Mark Mitchell]

What I did was set up a script in the cron to delete files when they hit over 30 days. The thing that can cause a problem is if they are being purged by creation date or last access date. I personnally went with access so that static files people used would not get deleted. Keep in mind that you have to take your backup into consideration in case it marks the files as accessed when it reads them. I can add more info if you think that this is something that you may want to try out.

[Bob Gulien]

Hi,

If you want to delete by timestamp, for instance everything but yesterday or everything but the last 2 days, you can use this:

find / -mtime +n -name '/dir/*' -print -exec rm {} ;

where +n = number of days back.
Between the quotes you can give the name or extension.