topic Re: Max arguments for rm command in Operating System - HP-UX

Max arguments for rm command

V2P — Tue, 27 Dec 2011 11:45:07 GMT

Dear Admins,

How do i determine the maximum arguments allowed in rm command? i.e number of files allowed in rm command to delete at a time.

SERVER1:/home/sit>ls -lrt|wc -l
34813
SERVER:/home/sit>rm *
sh: /usr/bin/rm: The parameter list is too long.

Regards,

V2P

Re: Max arguments for rm command

rariasn — Tue, 27 Dec 2011 12:01:42 GMT

Hi:

[E2BIG] The number of bytes in the new program'sargument list plus environment is greater
than the system-imposed limit. This limit isat least 5120 bytes on HP-UX systems.

man 2 exec

rgs,

Re: Max arguments for rm command

Dennis Handly — Wed, 28 Dec 2011 05:36:09 GMT

>How do I determine the maximum arguments allowed in rm command?

You don't. You have your plan A which is just "rm *".

Note: A closer estimate would be: { env; ls; } | wc -c

If you have a total length of all of the files about 2 Mb, then you switch to plan B:

find . -type -f -exec rm -f {} +

(Or you always use plan B.)

>This limit is at least 5120 bytes on HP-UX systems.

It is about 2 Mb and "-exec ... +" knows what it is, compared to dumb xargs(1).

Re: Max arguments for rm command

V2P — Tue, 27 Dec 2011 12:39:53 GMT

Hello Denis,
SERVER1:/home/sit>ls |wc -c
2022086
Bytes would be 2022086
SERVER1:/home/sit>ls |wc -l
31707
Here am able to delete the 31707 files with rm*. Is this the same calc u meant?

Re: Max arguments for rm command

V2P — Tue, 27 Dec 2011 12:40:24 GMT

Hello rariasn,
SERVER1:/home/sit>ls |wc -c
2022086
Bytes would be 2022086
SERVER1:/home/sit>ls |wc -l
31707
Here am able to delete the 31707 files with rm*. Is this the same calc u meant?

Re: Max arguments for rm command

Dennis Handly — Tue, 27 Dec 2011 13:48:28 GMT

> ls | wc -c 2022086
>Bytes would be 2022086
>Here am able to delete the 31707 files with rm *. Is this the same calc you meant?

I'm saying that 2 Mb may be too large to pass as arguments.

Re: Max arguments for rm command

V2P — Tue, 27 Dec 2011 13:18:34 GMT

But here am able to delete the files with rm * command. Then what parameter manages the rm max args?

Re: Max arguments for rm command

rariasn — Tue, 27 Dec 2011 13:39:29 GMT

Hi:

http://h30499.www3.hp.com/t5/General/chmod-command-on-long-list-of-files/m-p/4239270#M136190

rgs.

Re: Max arguments for rm command

Dennis Handly — Tue, 27 Dec 2011 13:51:00 GMT

>But here am able to delete the files with rm * command. Then what parameter manages the rm max args?

Ok, I found the limit is more like 2 Mb:

http://h30499.www3.hp.com/t5/System-Administration/usr-bin-cp-the-parameter-list-is-too-long/td-p/4449811

$ getconf ARG_MAX
2048000

This is a fixed value.

Re: Max arguments for rm command

Bill Hassell — Wed, 28 Dec 2011 02:01:48 GMT

The correct answers are:

There is no limit to the number of files that can be deleted, but

there is a limit to the number of characters on the command line

So for the specific example: rm *

The star * is a special character for the shell and for default environments, it will effectively rewrite the command line with rm plus the names of all the files in the current directory. If every file is exactly two letters (as in aa, ab, ac, ad...Aa, Ab, Ac...11, 12, 13,...) then about 666,000 files can be removed with the rm command (2 characters plus 1 space).

So it has nothing to do with the number of files, it has to do with the length of the resultant command line. It is not a good idea to think of managing massive numbers of files with simple file matching characters like "*". The first consideration is what can you do to stop the madness of having thousands of files in one directory. HP-UX has no problems with thousands, even millions of files in one directory. But the fact that it can be done does not make it a good idea. Most of the massive directory problems I have seen have been due to bad designs trying to create a database with files rather than using a real database.

Now if you are forced to deal with a bad design that will not be fixed, the next best is to use command lines that will work and avoid those that fail or take enormous amounts of time (and disk I/O). Assuming your current directory has 1 million files, the first commands to avoid are simple ls and ll commands. The result will scroll off the top of your screen, onto the floor and down the stairs...

Instead, you limit the listing to what you are really looking for. If you are looking for files that start with the characters abc and end with 123, then limit the match: ls abc*123 . If you need to remove all the files in a directory (with no subdirectories), then use rm -r mydirname and *not* rm mydirname/*

If the directory has subdirectories, you'll have to be more creative by breaking the filenames into manageable groups.

Re: Max arguments for rm command

Dennis Handly — Wed, 28 Dec 2011 05:34:28 GMT

>the next best is to use command lines that will work and avoid those that fail or take enormous amounts of time (and disk I/O).

What's why plan B should be used.

>the first commands to avoid are simple ls and ll commands. The result will scroll off the top of your screen,

You can always redirect the output.

>If you need to remove all the files in a directory (with no subdirectories), then use rm -r mydirname and *not* rm mydirname/*

Right. And copy the permissions/owners if needed to recreate the directory.

>If the directory has subdirectories, you'll have to be more creative by breaking the filenames into manageable groups.

Plan B doesn't have any issues with this, unless you don't want to remove subdirectories. And then with the appropriate -prune, that will work too.