large_ncargs_enabled : arg list too long : ARG_MAX=2048000

[Eric Sun]

I am processing large a mount of files in 11.23, but the current ARG_MAX is 2048000, therefore I ran into "ksh: /usr/bin/ls: arg list too long" problem a lot.

I am wondering if there is a way to tune the kernerl paramter to increase the total length of the arguments for exec() in bytes, including environment data.

Or if there is a way to disable the limit of ARG_MAX?

Your advice is greatly appreciated.

E.S.

HP-UX hp14 B.11.23 U ia64

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=765004

large_ncargs_enabled

I did not see this parameter in 11.23 either.

[TwoProc]

I'm going to give you a different answer, than what you asked for, but it would still get you there, if you're willing to make a small change. However, if you what you're really wanting is the way to change arg_max - then please ignore the reply.

Usually the way the process a long list of arguments is to use the "xarg" command.

An example

> file *
Arg list too long...

use instead:

> ls * | xargs file

The above command is great if the rm command is not going to need arguments other than what you're supplying via the wildcard. It won't work if you're going to pass a switch command to rm (like -f or -i).

So, to fix that you'd use the "{}" placeholder like in the "find" command.

> ls *| xargs -i file {}



If you watch the above commands from "ps" you'll see that the "file" command is going to be reissued with a new set of arguments over and over again until the whole list is finished.

[Sandman!]

xargs would be the way to go...man xargs(1) and look-up its '-n" option also.

cheers!

[Eric Sun]

Thanks. I know xargs is a good workaround, but I will just have to promote a bunch of code changes to the production.

Appreciated.

[Bill Hassell]

You'll never be able to make it large enough. If you could increase it to 20megs, you'll eventually need 25megs, and so on. The command line used to be just 2048 characters max and everyone used xargs (or better yet, redesigned massively large directories which are a real sysadmin pain to manage). There is no simple way to predict just how big the expanded list will become.

[A. Clay Stephenson]

I don't ever remember 2K but 1 definitely remember the limit at 5120 bytes because that was the SVR2 guaranteed minimum argument size and you simply wrote code that lived within that limit. Any values above the minumum were gravy. You really need to rewrite you code to use xargs; that's what it's there for. Of course, you could compile you own shell and set it to whatever limits you like but the far smarter play is to assume a MUCH smaller arg_max and live within those boundaries.

[Eric Sun]

Greatly appreciated for all the above suggestions. I will adopt xargs.

THX everyone.