More Time

[Rob Blake]

I'm trying to figure out if there is a way to write a ksh script that will find and delete all directories and their sub-files that are more than 5 minutes old. Anyone have an idea how/if I can use mtime or ctime or something like that to make something that works? Basically a clean-up type of ksh script to put into crontab to run every 5 min.

[Ivan Ferreira]

Hi Rob, ctime and mtime uses multiples of 24 hours.

[Pete Randall]

Rob,

You need to look at a combination of the touch and find commands. Use touch to create a file with a time stamp of 5 minutes prior. Then use find to detect and remove any files older than your reference file.

touch -t [[CC]YY]MMDDhhmm[.SS] ref_file
find /start_dir -type f ! -newer ref_file -exec rm {} \;
-or-
find /start_dir -type f ! -newer ref_file | xargs rm


Pete

[Victor Fridyev]

Hi,

You can use command
touch -t time file_stamp
time can be set up somehow as now -5 minutes
After you run
find dirname -newer file_stamp -exec rm {} \;

HTH

[James R. Ferguson · Accepted solution]

Hi Rob:

First compute a timestamp that is 5-minutes less than the current time:

Use the following perl script called "/tmp/time":

#!/usr/bin/perl
$t=((time)-300);
($mon,$day,$hr,$min)=(localtime($t))[4,3,2,1];
$mon++;
printf("%02d%02d%02d%02d\n",$mon,$day,$hr,$min)

In your cron task do:

# TS=`/tmp/time`

# touch -amt ${TS} /tmp/myref
# find /path -xdev -type f ! -newer /tmp/myref | xargs rm
# find /path -xdev -type d -newer /tmp/myref | xargs rmdir
exit 0

This does a pass for files first, and then secondarily for empty directories. Since deletion of the files in a subdirectory will update the directory's timestampe ('mtime') I chose the positive form of '-newer' whereas the negated '-newer' argument was applied to the files.

Regards!

...JRF...

[Arturo Galbiati]

Hi,
I use this command to remove teh core file:
It founds the core file in the current modified from 24 hours ago to now

find . -name "core" -mtime -1 -type f -exec rm {} \;

I scheduled it in crontab to run every 5 minutes and teh result is that I remove teh core file every 5 minutes.
HTH,
Art

[Stephen Keane]

Or (proving yet again that there is more than one way to skin a cat in shell) ...

Your cron job (which runs every 5 minutes) should look for a reference file in a known location.

If the file is absent, then create it (its timestamp will be set to now) do nething else this time.

If the file is present, then it was created by your cron job 5 minutes ago, so any file older than it can be removed. Then touch your reference file (its timestamp will be set to now) so that the next time the cron job runs it will be 5 minutes old.

The advtange of doing it this way is you can change how old the files need to be before deleting them by changing the cron job frequency rather than having to edit the script each time.

Just my £0.02 worth.

[Rob Blake]

JRF suggests:

# find /path -xdev -type f ! -newer /tmp/myref | xargs rm
# find /path -xdev -type d -newer /tmp/myref | xargs rmdir

James, just looking at it, the above seems to delete all files that are younger than the reference file /tmp/myref rather than older.

Am I reading this wrong? What can I put on the end to replace the xargs rm and xargs rmdir that will show me which files are fixing to be deleted before I actually try and run a test of this.

So far, the script looks like this ...

rm /tmp/myref
START=${HOME}
echo ${START}
TARGET=${HOME}/targetdir
echo ${TARGET}
cd ${TARGET}

#####
# Run temp_time to get the date time less 5 minutes
#####
TS=`${PS_HOME}/temp_time`

#####
# Touch to create a reference file in tmp directory
#####
touch -amt ${TS} /tmp/myref

#####
# find all files and directories in the target directory that are older
# than the reference file.
#####
find ${TARGET} -xdev -type f ! -newer /tmp/myref | xargs rm
find ${TARGET} -xdev -type d -newer /tmp/myref | xargs rmdir

#####
# Exit script
#####
cd ${START}
exit 0

[Pete Randall]

The "! -newer" syntax says not newer, i.e. older.


Pete

[Stephen Keane]

You do realise that when you delete a file from a directory, you update the directory (and hence its access time) as well?

[Pete Randall]

Sorry, I missed your second question.

To test, replace the "xargs rm" with "xargs ll".


Pete

[Stephen Keane]

Which won't change the access time, so isn't a perfect test.

[James R. Ferguson]

Hi (again) Rob:

Yes, Pete is correct, the "!" symbol is negation, so "! -newer" is "not newer" or hence, older.

Yes, of course, deleting a file from a directory updates the directory's *modification* ('mtime'). Traversing (reading) a directory with 'find' updates the directory's lastaccess time ('atime').

By default the '-newer' argument to 'find' operates on the structures 'mtime'.

The idea I had was to make a pass to delete *files* matching your criteria and then in a second pass delete empty directories. The 'rmdir' will fail if the directory isn't empty which is what you want if files have been *added* anyway.

Regards!

...JRF...

[James R. Ferguson]

Hi Rob:

I must chuckle. Stephen's comment about there's more than one way to do anything is so appropriate!

Computing a file's timestamp a few minutes ago is so easy that I focused on that approach rather than a continuous crontask that simply creates its reference point the first time and nothing more.

Stephen deserves high marks here!

In any event, to be perfectly correct at the turn of a year, I should offer:

#!/usr/bin/perl
$t=((time)-300);
($year,$mon,$day,$hr,$min)=(localtime($t))[5,4,3,2,1];
$year+=1900;
$mon++;
printf("%04d%02d%02d%02d%02d\n",$year,$mon,$day,$hr,$min);

Calling the above script whatever you want (e.g. $HOME/time5) you can abbreviate your shell's creation of a reference file to:

# touch -amt `$HOME/time5` /tmp/myref

Regards!

...JRF...

[James R. Ferguson]

Hi (once more):

I'm sorry, I just noticed your other question, "What can I put on the end to replace the xargs rm and xargs rmdir that will show me which files are fixing to be deleted before I actually try and run a test of this?"

Simply write "...|xargs" without any other arguments. 'xargs' will substitute 'echo' in the absence of any argument.

Another useful feature of 'xargs' is to add a '-t' option to cause a trace of what was done to be printed:

... | xargs -t rm

Regards!

...JRF...

[Rob Blake]

Thanks for all the great help and ideas. So far the resulting two(2) scripts have been created and tested manually against our development environment:

SCRIPT: temp_time
#!/usr/bin/perl
$t=((time)-300);
($mon,$day,$hr,$min)=(localtime($t))[4, 3, 2, 1];
$mon++;
printf("%02d%02d%02d%02d\n",$mon,$day,$hr,$min);
exit;

SCRIPT: clear_up
#!/bin/ksh
#####
# Remove old reference file
#####
rm /tmp/myref

#####
# Setup home and target directories
#####
START=${HOME}
TARGET=

#####
# Run temp_time to get the date time less 5 minutes
#####
TS=`${START}/temp_time`

#####
# Touch to create a reference file in temporary directory
#####
touch -amt ${TS} /tmp/myref

#####
# find and delete all files, directories, and
# sub-files older than the reference file.
#####
find ${TARGET} -xdev -type f ! -newer /tmp/myref | xargs rm
find ${TARGET} -xdev -type d -newer /tmp/myref | xargs rmdir

#####
# Exit script
#####
cd ${START}

exit 0

Will be copyng these to the pre-production environment to test maually and via crontab of clear_up script before moving to Production. Thanks to everyone for their input and assistance on this. Believe me - the UNT students will appreciate it.

[James R. Ferguson]

Rob:

I suggest you use the second perl script I posted which covers the change of the year.

Glad to have been of help...

Regards!

...JRF...

[Rob Blake]

james - thanks for the advice. Changes made as suggested to temp_time.