rm -r command is very slow

[CSG Office]

Hi Everyone,

I just saw this question in the humor section of questions you do not want your System Administrator to ask. Fortunately, I am not running this command from /:)

I created a 113GB partition on our newly purchased StorageWorks SAN. The area I created is /SANDISK/vol1. I went ahead and populated this area with about 80GB of junk data (tons of files) to do some tests. The access time is excellent and I am now ready to move the real data that I want there. I decided to clean up the area. I know that I could have just unmounted the area and done a newfs on the raw device file, but I decided to use the "rm" command instead. Why the heck does it take so long to complete? I did the following:

%cd /SANDISK/vol1
%rm -r *

This command took 4 hours to complete. A newfs would probably have only taken about a minute to complete. The problem is that newfs wipes out everything, and sometimes we have to remove large amounts of data in pieces. Is there something I can do to speed this up?

Thanks,

Brian Whyte

[Victor BERRIDGE]

Hi Brian,
I have also noticed the phenomenon when on large filesystems with numerous files and I have opted for recreating a new FS when I know I dont want anything on the old FS...
My ratio was 30 min vs 2 min (well it takes me 30s already to type...)
something else I noticed was the CPU consumption which made me favor for remove and newfs...

All the best
Victor

[Sridhar Bhaskarla]

Hi Brian,

Unless you run multiple rm commands in the same directory, it's not possible to speed up as it has to unlink so many files but one by one. It has little to do on how fast your disk system is. I would do something like this.

%cd /SANDISK/vol1
%rm -rf dir1&
%rm -rf dir2&
%rm -rf dir3&

And see how long it takes now.

-Sri

[Mark Greene_1]

Were all or most of the 80GB of files in a single directory? If so, each remove would force an update and rewrite of the directory.


mark

[CSG Office]

Hi All,

Just to add, I do not think that we have any hard links. I have never created a hard link. Not sure that it would matter this much anyway.

Thanks,

Brian Whyte

[CSG Office]

Hi,

Goodness! You guys are quick. I could try removing directory by directory, but since there are a 100 directories, I'd rather not type that command 100 times.

Regards,

Brian Whyte

[Tim Killinger]

I suspect you had huge numbers of files in a single directory, where the directory itself must be resorted and written many,many times. This is definitly the case with the VMS OS.

You could try writing a listing of the files to a text file, then sort the file alphabetically descending - place a "rm" on each line and execute it as a script. In VMS, this would make the process go from hours to minutes.