Removing a directory with large number of files

[James Brand]

I have a directory in a vxfs filesystem with over 2 million small files in it. Using
"rm -rf dirname" is currently removing about 27,000 files/hr. At this rate it will take several days to finish.

Can anyone suggest a faster way, or a method of deleting a non-empty directory?

[Matti_Kurkela]

A single directory with a huge number of files is just about the worst possible case for vxfs. The only workaround would be to unmount and re-mkfs the entire filesystem, but that is possible only if you can afford to lose everything on that filesystem (or if it's easy enough to backup or move away the things you *don't* wish destroyed).

If the existence of the directory is blocking you from doing other operations, you might wish to rename the directory before deletion, so that it does not interfere with your other activities: renaming a directory is just a single operation, so it should always be fast. Then you can leave the directory removal operation running in the background with nohup and nice, so it won't block you from logging out nor spend any CPU power that would have more important uses.

nice nohup rm -rf /some/where/hugedirectory &

mK

[James R. Ferguson]

Hi James:

The recursive remove you are using is probably about as fast as it is going to get.

If you only need a small fraction of the files within the filesystem, it might be faster to copy those somewhere; 'newfs' the filesystem; and restore the files you intend to keep.

Very large, very flat filesystems like you have lead to very long times for file removal.

Regards!

...JRF...

[Steven E. Protter]

Shalom,

It is going to take several days to finish.

That is what you get for putting 2 million files in a vxfs file system.

ls -1 > list
while read -r fn
do
rm -f $fn
done < list

That might be faster.

SEP

[Lijeesh N G_1]

Hi,

Better to recreate file system(newfs) after copying or backup required files.

Regards,
LIJEESH N G

[James Brand]

Recreating the filesystem is not an option, application outages are hard to come by. I will just let the delete run in background with nohup for as long as it takes.

I've already scolded the application team that created this mess and they have fixed their process.

Thanks to all!