rcp - The parameter list is too long

[ikbea]

In HPUX 10.20,
$rcp -p hostname:/home/a/b/c/* /home/d/e/f
sh: /usr/bin/rcp: The parameter list is too long

There are about 1000 files in directory c, the filename: 1, 2, 3 .... 1000

#getconf ARG_MAX
20478

I would like to know how to determine the maximum length/arguments in single command ?
Thanks

[steven Burgess_2]

Hi

A quick work around would be to define a list of files that you need to copy then do them in a loop

for f in `ls -1` /home/a/b/c/*
do
rcp -p $f hostname:/home/d/e/f
done

Or

Install patch, which will fix your problem


http://www5.itrc.hp.com/service/patch/patchDetail.do?BC=patch.breadcrumb.main|patch.breadcrumb.search|&patchid=PHKL_16751&context=hpux:800:10:20

HTH

Steve

[Siem Korteweg]

Use the -r option of rcp:

rcp -pr hostname:/home/a/b/c /home/d/e/f

If you do not want to rcp sub-directories:

cd /home/d/e/f
remsh hostname "cd /home/a/b/c; find . -type f | grep -v '/.*/' | cpio -oca" |
cpio -ic

[ikbea]

In swlist, the patch PHKL_16751 is installed.

[hein coulier]

tar can be an alternative :

cd ${source_dir}
tar cf - '*' |
remsh ${target_host} "cd ${target_dir} ; tar xpf -"

[ikbea]

How to get value of max. length/ max. arguments in single command ? By "getconf ARG_MAX" ?
Any parameter can be set in order to increase the max. value ?

Thanks for kindly help

[Mark Grant]

It's a function of the shell so it would depend on which shell you were using.

[ikbea]

/usr/bin/sh POSIX shell

[Robert-Jan Goossens]

Hi,

Check if kernel has the parameter " large_ncargs_enabled = 1 " enabeled, default is 0.

This would raise the ARG_MAX value to 2048000.

Hope this helps,
Robert-Jan

[Alex Ostapenko]

My response doesn't exactly answer your question, but maybe this observation will give you a little insight to the behavior of "rcp" command.

Situation #1 (your situation) -- If you are currently logged into your TARGET system and you are copying remotely from the SOURCE system, e.g., rcp -p SOURCE:/home/a/b/c/* TARGET:/home/d/e/f, the SOURCE system wildcard character * will expand the parameter list to:
SOURCE:/home/a/b/c/1
SOURCE:/home/a/b/c/2
...
SOURCE:/home/a/b/c/1000
for each of your thousand files. As the filenames are fully-specified, you are sucking up a whole bunch environmental space just for the filenames.

SITUATION #2 -- You login your SOURCE system and copy locally from SOURCE remotely to the TARGET, e.g., rcp -p * TARGET:/home/d/e/f, assuming you have already cd'd into /home/a/b/c. In this case, the wildcard will expand to simply:
1
2
3
4
...
1000
using up a lot less environmental space.

Personally, I like the "tar-ball pipe" solution the best as described by a previous response. As long as the tar-ball is created on the SOURCE system, then piped to the TARGET system, it will be faster and not be subject to limitations due to the number of files because only 1 big file (the tar-ball) is being transferred across the network instead of 1000 small files. Overhead of opening, copying, closing of each individual file adds some time to the transfer time. You can even throw in a "compress" or "gzip" into the pipe to further reduce the amount of data transferred across the network. Here is complete pipe assuming you're currently logged in on the TARGET system:

remsh SOURCE -n "cd /a/b/c; tar cf - . | gzip -c" | (cd /d/e/f; gunzip -c | tar xvf -)

Happy remote copying!

=:-) Alex

[A. Clay Stephenson]

The solution to your problem is the xargs command. It should be used in any case where it is possible to overflow ARG_MAX. The idea is that xargs takes a few fixed arguments and combines them with a list that is divided into chunks. Man xargs for details and examples.

[Bill Hassell]

AS mentioned, ARG_MAX controls the size of a command line. Since the * character is translated by the shell into a list of all the filenames in the directory, the command line may be megs in length. There was a patch for the obsolete 10.20 system to increase the ARG_MAX to a couple of megs which may help but as Clay mentioned, you should always use xargs when there are unlimited line lengths generated by filename generation (ie, the *).

[Siem Korteweg]

In the case of rcp, xargs will not be of help, as the destination of the copy has to be the last argument of rcp. xargs supports only initial arguments of the command to be started.