/usr/bin/tar: The parameter list is too long

[Ravikumar_2]

Hi,

I'm using HP-UX 10.20. I've around 415156 files in a directory and want to take backup of specific files. when I tried to ls or tar the directory, it failed with the follg message.
********************
# tar cvf ttjune.tar *jun*
sh: /usr/bin/tar: The parameter list is too long.
********************
Can someone help me on how to do the backup?

Many thanx in advance.

[Danny Ramos_1]

hi,

tar is only capable of archiving a maximum of 2GB. try to execute the same command but decreasing the list for backup. *jun* may exceed 2 gb.

hope this helps.

regards,
danny

[Ravikumar_2]

Thanks for the reply. I could not even liat(ls) few files in that directory. Is there any limits on the no. of files in a directory?

FYI, total size of the directory is around 22GB with 415156 files. Each file size may be between 1 to 20Mb.

Thanks.

[Michael Tully]

You have two problems.

The first is that your ls list is too long.

Try this command to confirm this

# getconf ARG_MAX
20478

If this is the result then you need to make changes to your kernel.

The second item is that 'tar' is not supported over 2Gb. The solution is to use a different backup command or get 'GNU tar'

[Michael Tully]

Further to my previous reply.

At a minimum you will need to install patch PHKL_16751 and any dependancies. This may not solve your problem, as there may still be a limit. Have a look at the posting from last year, which has has a similar problem.

http://bizforums.itrc.hp.com/cm/QuestionAnswer/0,,0x5188abe92dabd5118ff10090279cd0f9,00.html

[Ravikumar_2]

Thanks Mike.

How to change ARG_MAX? Is it dependent on other parameters?

Regards

[Michael Tully]

The patch I mentioned will make the necessary changes. You will need to install it and any dependancies. When you get the patch it will show you which patches are the dependancies. Of course you'll need an outage for this.

[Michael Tully]

One further thought.

Make sure that you create an ignite (make recovery) tape before doing any kernel changes.

# make_tape_recovery -x inc_entire=vg00 -I -v -a /dev/rmt/?mn (note the use of the no-rewind command). Also here is a link to where you can the latest ignite software.

http://www.software.hp.com/products/IUX/download.html

[Christian Gebhardt · Accepted solution]

You can list or tar files without worrying about the number of files with command find:

touch timestamp
tar cvf ttjune.tar timestamp
find . -name "*jun*" -exec tar uvf ttjune.tar {} \;

Chris

[Ravikumar_2]

Thanks a lot for all. I can use find to solve my problems.

[Balaji N]

Hi,
This problem is due to the no of arguments passed to tar exceeding the maximum limit as specified by ARG_MAX.

This is defined in limits.h though dont exactly now how to change this.
An alternate workaround for this problem to reduce the arguments to tar.

Use some thing like
tar cvf ttjune.tar [a-c]*jun*
and then
tar rvf ttjune.tar [d-f]*jun*
and so on.

Depends on how your files are named.

Alternate and easy option is to write a script to move these files to another directory (a script because "mv *jun* /june" will also fail) and then just tar the entire directory.

HTH
-balaji

[Seth Parker]

Check SAM and see if you have the "large_ncargs" kernel parameter listed. We had to turn this on for our 10.20 box because of the large numbers of files we had in some directories.

Not sure if this will be enough though. That's a lot of files....

Check out this link for more information about this parameter:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xe58b8cc5e03fd6118fff0090279cd0f9,00.html

Good luck!

[paul fritzsche]

try to do tar on the catalogue where the files are in. I had the exact same problem a while ago and solved it like that

[Ravikumar_2]

Thanx guys.

Paul, Could you pls elaborate on that? Thanx.

[paul fritzsche]

ahhh.. I see now, you only wanted to take backup of specific files? do a new catalogue, copy those specific files to that catalogue and then do a tar on the catalogue instead of the files.

ie

mkdir jun
cp *jun* jun/
tar cvf jun.tar ./jun

this should do the trick?

[Balaji N]

paul,

wont that give the same error (parameter list is too long) when attempting that copy?

-balaji

[paul fritzsche]

oh, yes it does. hmm..

then you have to make a script..

make a directory somewhere..
mkdir /tmp/delete

Then you make a logfile of all the entries you want to copy..
xargs ls -1 *jun* > /tmp/delete/delete.log (or something)

wait until the file has been created and then make a script with vi


-----
for i in $(CAT /tmp/delete/delete.log)i do
cp -p /"where your files are"/$i /tmp/delete/

done
-----

then wait again until all the files have been copied to that loction and do the tar thingie I wrote to you earlier

This should do the trick (I hope =)

[paul fritzsche]

Oh, forgot half of it =)

before doing the tar thingie, yo have to save the script ie
copy.sh

and then do a chmod on it
chmod 755 copy.sh

to make it executble

start it with
./copy.sh

and then sit back and wait for all the files to be copied and THEN do the tar-thingie =)