topic Re: Removig large directory in Operating System - HP-UX

Removig large directory

HrishiMV — Sun, 14 Sep 2008 07:10:34 GMT

Dear Masters,

OS HP-UX 11.11

I wanna remove one directory which is having application logs more than 400000 andsize 20GB+
I tried rm -rf command for remove logs directory but its taking to much time.

Please suggest me any solution which will took 2-3 Hrs

Re: Removig large directory

SKR_1 — Sun, 14 Sep 2008 07:50:42 GMT

Why are you removing this application log directory?

You should clear the logs inside this directory if you dont need that logs.

> log file name ( to trim the log file )

rm -rf directory ( to remove the directory )

Thanks

SKR

Re: Removig large directory

SKR_1 — Sun, 14 Sep 2008 07:52:15 GMT

you have created three threads for the same question, please close the other two threads.

Thanks

SKR

Re: Removig large directory

HrishiMV — Sun, 14 Sep 2008 08:57:01 GMT

Dear SKR

We tried rm -rf directory command. But it is taking time in hours. But Is there any faster way? Is it possible to change any kernel parameter for quicker result?

Re: Removig large directory

SKR_1 — Sun, 14 Sep 2008 09:46:24 GMT

No, you dont have any option for any change in any kernal parameter for this type of task

Thanks
SKR

Re: Removig large directory

Hemmetter — Sun, 14 Sep 2008 11:44:51 GMT

Hi

to rm a file a lot of things have to be done:
e.g.
maintaining file system journal, change filesystem blocklist, change directory entry, commit journal.
this has to be done for each file and for millions of files .... a lot of work.

Think of a cleaning job that runs more frequently.

Think about splitting the logfiles into different directories by application or date.

Also think about a dedicated filesystem. Then you can play with vxfs-mountoptions:
see mount_vxfs(1m).

Generally it is a bad idea having that much files in one directory. Every "ls -l" causes huge load. Think about all that poor programmed scripts with lots of "ls -l | awk ...". And if that directory is shared via nfs/cifs situation gets worse.

rgds
HGH

Re: Removig large directory

Hemmetter — Sun, 14 Sep 2008 12:38:52 GMT

hi,

and what about renaming the directory "away", recreate it for new logs and then slowly delete the old?

rgds
HGH

Re: Removig large directory

Duncan Edmonstone — Sun, 14 Sep 2008 13:53:47 GMT

Hi,

I guess if the filesystem the directory is in is a VxFS filesystem you could try remounting it temporarily with the tmplog option to reduce the amount of metadata logging VxFS has to do. Something like this should work:

mount -F vxfs -o remount,tmplog /myfilesystem

Obviously substituting your filesystems name. Once complete you should be able to reinstate the original filesystem options by using:

mount -F vxfs -o remount /myfilesystem

This is all assuming your filesystem is in /etc/fstab. I also have no idea whether this will work on a system without OnlineJFS, and whether it will make any difference at all to the performance of the "rm -rf" command - you'll just have to give it a try...

HTH

Duncan

Re: Removig large directory

Michael Steele_2 — Sun, 14 Sep 2008 14:16:29 GMT

Hi hrishiMV

This is normal behavior that can be checked for progress with a find command.

find /filesystem | wc -l

Re: Removig large directory

Dennis Handly — Sun, 14 Sep 2008 16:06:48 GMT

If you don't care about the 20 Gb, you can just unlink the directory.

I suppose emptying each file (as SKR says) and then unlinking the directory wouldn't lose most of the space.

If there is nothing else in the filesystem, you could just recreate it.

Re: Removig large directory

Suraj K Sankari — Mon, 15 Sep 2008 04:52:01 GMT

Hi,

If you want to delete the directory then first trim the logfiles with
> logfile (trim the log file)
once the files got trimed then apply the rm -Rf directory, it will remove your directory more faster then only doning rm -Rf

Suraj