Operating System - HP-UX
1751976 Members
4981 Online
108784 Solutions
New Discussion

Unable to delete 10 million tiny files in a directory

 
NDO
Super Advisor

Unable to delete 10 million tiny files in a directory

 

Hi 

 

Unfortunately I have run out of inodes, on my file system, in which there is a subdirectory with almost 10 million tiny files (3 digit size).

I have created a script with two references (Jan to March), the script seems to be working, but at the same time, someone is sending files to the very same directory.

Can someone help me into delete files in a faster way......

9 REPLIES 9
Dennis Handly
Acclaimed Contributor

Re: Unable to delete 10 million tiny files in a directory

The only faster way is to delete the whole filesystem.  ;-)

Or next time you create them, tar them up into much bigger files every so often.

NDO
Super Advisor

Re: Unable to delete 10 million tiny files in a directory

 

 

I did not create them, they are being sent from another system. If I boot in single user mode, and just mount that file system, will "rm -rf" being faster?

Dennis Handly
Acclaimed Contributor

Re: Unable to delete 10 million tiny files in a directory

>they are being sent from another system.

 

Create a local job that tars the up when finished.

 

>If I boot in single user mode, and just mount that file system, will "rm -rf" being faster?

 

I'm not sure that will help much, is there any other activity on that filesystem?

 

 

NDO
Super Advisor

Re: Unable to delete 10 million tiny files in a directory

Nothing is going on in that file system, apart from that directory being receiveing files (3 digit size files).

Last night I try to 

tar cvf optima.tar *.csv

 But that also failed.. 

 

But if tar were successfully would´nt be the original files still be sitting there?

Dennis Handly
Acclaimed Contributor

Re: Unable to delete 10 million tiny files in a directory

>tar cvf optima.tar *.csv
>that also failed

 

Command line too long?

 

>if tar were successfully wouldn't be the original files still be sitting there?

 

Yes, you would continually remove them as you created each tarfile.

(All this assumes you want to keep the data.)

 

>two references (Jan to March), the script seems to be working

 

I forgot to ask, are you using the faster: find ... -exec rm -f {} +.

Or just the slow: find ... -exec rm -f \;

NDO
Super Advisor

Re: Unable to delete 10 million tiny files in a directory

 

 

I am using the 

find . -type f -mtime +30 -exec rm -f {} +

 as command line, and in the script I am also using 

-exec rm -f {} +

 

Dennis Handly
Acclaimed Contributor

Re: Unable to delete 10 million tiny files in a directory

>find . -type f -mtime +30 -exec rm -f {} +

 

Then you can't improve that find/rm.

NDO
Super Advisor

Re: Unable to delete 10 million tiny files in a directory

You mean, I should have either the script running or just the find/rm command? Not both?

Dennis Handly
Acclaimed Contributor

Re: Unable to delete 10 million tiny files in a directory

>You mean,

 

I only meant that your find/rm is optimal and not the poor -exec rm -f \;