Removing HUGE amt of files

[Ratzie]

We have a directory that holds files from a upstream system that errored for some reason.
Basically, files .xml that were not accepted from another system was placed in this /bad directory.

We were under the impression that this directory was purged. But, we are wrong. Now it is so huge that wc -l, will not return anything. We need to delete files older then 2 months. But, our find command hung as well.
Any ideas?

We wanted to keep a log of what was deleted as well.
This is our command..
We first touched a file with a date of 2 months ago.

find . -type f -newer ../testfile -exec rm -f {} ";"

[Victor BERRIDGE]

Hi,
I think the use of xargs with your find will solve your problem

Good luck
All the best
Victor

[Pete Randall]

The first thing I notice is that your find command is going to remove the files that are LESS than two months old. You would need to do "find . -type f ! -newer ../testfile -exec rm -f {} \;".

Secondly, are you sure the command hung, or is it just gradually working it's way through all the files?


Pete

[Jean-Luc Oudart · Accepted solution]

We use some cleanup scripts that delete files with specific suffix older than XX days

we still use the -exec rm to remoce them but we use xargs -l10 to log them 1st into a log file (prior to deletion).
the "-l10" will reduce the number of processes generated.

##########################################
ind_PRT () {
DIR=
if [ -d $DIR ]
then
echo $DIR
echo "================================================="
find $DIR \( -name "*.[SP]RT" -o -name "*.PRT.lan" \) \
-mtime $DAYS | xargs -l10 ll echo "deletion..."
find $DIR \( -name "*.[SP]RT" -o -name "*.PRT.lan" \) \
-mtime $DAYS -exec rm {} \;
else
echo " directory does not exist for $branch"
echo "================================================="
fi
}

#####################################
Regards
Jean-Luc

[Jeroen Peereboom]

You really should consider find ... | xargs rm -f.
This will call the rm command as few times as possible (with each time as many as possible arguments).

On the other hand, really many small files slow down the removal of the files from the filesystem.

JP.

[Ratzie]

Thanks