topic Re: 'ls' command showing argument too long !! in Operating System - HP-UX

'ls' command showing argument too long !!

Chris Fung — Mon, 08 Sep 2003 16:34:39 GMT

Hi All,

I just encountered a problem when running the 'ls -R /a/b/c/*' command .....and the system complained argument too long....

However, if I just use 'ls -F /a/b/c' and it works fine......also two different accounts run the first command will have different results....one was alright while the other complains argumen too long...

I am just wondering what's wrong with 'ls'...is there any buffer or environment variable I can set to get rid of this problem ??

FYI, I am using KSH

Many thanks,

Chris

Re: 'ls' command showing argument too long !!

Pete Randall — Mon, 08 Sep 2003 16:38:14 GMT

Chris,

When using the wild card, the shell attempts to expand the file name using file name completion and the list simply ends up being too long. There is no way to change this shell behaviour, however, you can work around it using find:

find /a/b/c |xargs ll

Pete

Re: 'ls' command showing argument too long !!

Steven E. Protter — Mon, 08 Sep 2003 16:39:12 GMT

My system posix and korn shell(ksh) has no problem with that command.

You may want to look and see if you have ls alias of some sort.

SEP

Re: 'ls' command showing argument too long !!

Massimo Bianchi — Mon, 08 Sep 2003 16:39:23 GMT

Hi,
there is a limit, i think 1256 bytes.

When you issue an

ls -R /a/b/c/*

the shell expands the * to the list of names. If they are, in lenght, more than XXX, you get argument too long.

ls -R /a/b/c/

do not cause the expansio, because there are no wildcard.

Massimo

Re: 'ls' command showing argument too long !!

Helen French — Mon, 08 Sep 2003 17:06:06 GMT

This has been discussed in these old threads:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x106937f45ef7d4118fef0090279cd0f9,00.html

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x5749c4c76f92d611abdb0090277a778c,00.html

Re: 'ls' command showing argument too long !!

Jeff Schussele — Mon, 08 Sep 2003 17:07:42 GMT

Hi Chris,

The difference between the -R & -F are as follows:

-R => list subdir contents as well as files.

-F => list subdir name & a symbol (/) that the entry is a subdir

The former will obviously have more entries, hence the "arg too long" msg

The account differences, I suspect, have something to do with ulimit differences.
The solution is to use the xargs (extended arguments) command.

HTH,
Jeff

Re: 'ls' command showing argument too long !!

Bill Hassell — Tue, 09 Sep 2003 02:41:50 GMT

This will ALWAYS occur if the number of files in the /a/b/c/ directory is very large OR the names of the files or directories are extremely long. As mentioned, the * character is very special. It is *NOT* passed to the ls command at all (under default conditions that is). Instead, the shell replaces the entire string /a/b/c/* with a full listing of all the names in that directory. To see this happen, change your command from:

ls -R /a/b/c/*

to

echo ls -R /a/b/c/*

and you'll see the same error. To see what the ls command sees in a smaller directory, do this:

echo ls /etc/*

Now tell the shell to stop expanding the * by escaping it's special meaning:

echo ls /etc/\*

and:

ls /etc/\*

The \ removes the special meaning of the * character. All this is covered in the manpages: man 5 regexp (the 5 is required, see man man)

So the error has nothing to do with ls but instead is a function of what the shell finds to match. For instance:

cd /a/b/c
ls -R

may work just fine because the expanded expanded list of matching filenames no longer contains the fullpath to this directory. However, add a few thousand more files to this directory and again you'll get the same error. When in doubt, use echo to see what your command line will look like after the shell finishes it's preprocessing.

---------------------------

For extra credit, here's a script that you can use to create 10,000 files to demonstrate the problem:

mkdir /var/tmp/dummy
cd /var/tmp/dummy

typeset -Z20 NUM=10000
while [ $NUM -gt 0 ]
do
touch IamAveryLONGfileNAMEtoSHOWcommandLINElimits_$NUM
let NUM=$NUM-1
done

Then use the * with either echo * or ls *

Warning: This will create a very large directory structure and will take a long time to create and also to remove.

Re: 'ls' command showing argument too long !!

Chris Fung — Wed, 10 Sep 2003 04:30:40 GMT

Hi Bill,

I still got one more questions.....!! Since different accounts (with different profiles) will produce different results for the "ls /a/b/c/*" command.

e.g. an account with a few lines .profile and environment variable defined will be able to execute the above command

While an other account with hundreds of environment variable defined are unable to execute the above command.

I am just wondering whether the environment variables set to the user account (i.e. the .profile) will affect the behaviour of the command or should say affect the maximum argument length of the Unix command. and Why ? Is that the shell environment has been assigned with a limited amount of buffer or some structures that both the environment variables and the command line share ?

Appreciated for your advice and input.

Cheers,

Chris,

Re: 'ls' command showing argument too long !!

Chris Fung — Wed, 10 Sep 2003 04:31:19 GMT

Re: 'ls' command showing argument too long !!

Chris Fung — Wed, 10 Sep 2003 04:32:26 GMT

Re: 'ls' command showing argument too long !!

Chris Fung — Wed, 10 Sep 2003 04:33:49 GMT

Re: 'ls' command showing argument too long !!

Bill Hassell — Wed, 10 Sep 2003 10:57:10 GMT

The "line too long" error is not from Unix (HP-UX), it is a shell error message. So the situation is depends on the shell. Most shells will use ARG_MAX (use the command: getconf ARG_MAX) to size their internal buffers. The user environment does not (normally) change this value (ie, no particular env param will change this limit or occupy this space).

What may be happening is that the argument list is different because of the specific directory permissions or (more likely) the ls command itself is not really the default built-in. For example:

alias ll='/usr/bin/ls -alF'

might be in a user's .profile so to see what a user 'really' is running, do this:

type ls

(type is an alias for whence -v) Unlike which and whereis, the type/whence command tells you exactly what will happen when you type a particualr string. Here are some examples:

type pwd ls ll for fc passwd

The ls -a option will return more filenames (ie, 'hidden' or dot-files). Do this in each user environment:

echo * | wc

This will tell you what * will return for a psecific user. Differences will be due to what the user is allowed to see based on user and group permissions.