Quikly remove large no of files

[Senthilkumaran.T]

Hi,
A large number of files (> 100,000 files) is in /backup directory. rm -fR /backup takes more than one day to clear all files. Can I use any other command to remove these files within a short time.

Thanks
Senthil

[Santosh Nair_1]

Senthil,

If /backup is a separate filesystem, I would recommend just recreateing the fs, i.e.:

umount /backup newfs -F vxfs
mount /backup

Hope this helps.

-Santosh

[Senthilkumaran.T]

Hi Santosh,

This is one of the directory in the file system. It is not possible to recreate the file system. Any other methods?

Thanks
Senthil

[Volker Borowski]

Senthil,

I guess there is no way out, if you can not recreate the filesystem.

If you need the directory, you could try to

mv /backup /backup_old
mkdir /backup

and than do a "rm -r /backup_old" in the background and let it run....
If you need this frequently, I would recommend to mount a seperate filesystem at /backup.

Volker

[Paula J Frazer-Campbell]

Hi
If your files names can be grouped then write serveral small scrips each one to remove a group and then cron these scripts to run one min after each other.

i.e.
-----------cut here-------------
rm a*
----------cut here--------------

This will spread the rm command accross the system - but be aware of load implications.

hth

Paula

[Wodisch]

Hello Senthil,

I guess the size of that directory itself (seen with "ls -ld")
is pretty big. And since UN*X (all UN*Xes) *always*
reads and writes directories at once, the whole dir will
be written every time a single file is deleted...

So, except for "mkfs"ing a new filesystem, you will have
to wait :-(

But then, immediately "rmdir" the directory, and "mkdir"
at new - the only way to shrink it.

HTH,
Wodisch

[Santosh Nair_1]

I second Volker sentiments. If it takes you over a day to clear out the files, then I would strongly recommend that you make that a separate filesystem as this could easily affect processes that might be using other files in that filesystem.

-Santosh

[Klaus Crusius]

Maybe it is an improvement, first to copy all other directories to a different file system, then re-create the original file system and copy back. You need the original file system unused during the operation.

Klaus

[Brian Markus]

I'm not sure if it would be faster this way, but it might be worth giving it a shot. How about a script that zero's out the files. then rm's them?

dir_to_delete=/bigdir
ll $dir_to_delete | awk '{ print $9 }' > $dir_to_delete/list_of_files.txt
for files in `cat $dir_to_delete/list_of_files.txt`
do
> $files
rm $files
done
exit 0

Something like that ??
Or what about the possibility of removeing the entry out of the superblock that refers to those inodes? (YUCK) but might be an option?

Just some thoughts.

Good Luck

Brian

[Peter Brimacombe]

1. create a new logical volume

2. make a new file system on it

3. mount /backup on the new logical volume

repeat steps 2 and 3 as needed

to get comfortable with this will take you a week, after that you can do it in 5 minutes