Tar and find command

[Phillip Popp]

Hi Running 10.2. I have two machines. I want to tar up files in a rather large directory older then say 3 years. I am combining the tar command and find command as follows. I can not tar the file to machine called hpserv01 as there is not enough space left on the hard drive, so I must build the tar file to the machine called hpserv02. When I try this command from hpserv01 it give me a return message of arg list too long. If I put in more then 1000 to reduce the size of the arguments it tells me o file, no data. Can some one give me some guidence on how to get past the list too long issue.


command from hpserv01: tar -cvf ./testdev_duts.tar `find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000`

any help would be greatly appreciated.

Thanks,

Phil

[James R. Ferguson]

Hi Phil:

Try this:

# find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | xargs tar -cvf ./testdev_duts.tar

Regards!

...JRF...

[Phillip Popp]

Thanks James. My only problem is that I then want to untar on hpserv02 and using the /net/hpserv01/LTX/hp_testdev_duts/ -mtime +100 will put it back where it came from when I untar. I eventually want to erase those files from hpserv01 when I have the data sucessfully untarred on hpserv02.

I would do a mv command but afraid I will loose my data and screw things up.

Any ideas ?

Thanks,

phil

[James R. Ferguson · Accepted solution]

Hi (again) Phil:

With 'tar' you should create archives with _relative_ paths instead of absolute ones unless (1) you want to "go nuts"; or (2) use 'pax' to strip the leading '/' to make a relative path.

That said, change you find to output relative paths:

# cd /path
# find . -mtime +1000

Regards!

...JRF...

[Phillip Popp]

Sweet!! that did the trick.

Thanks.

Phil

[Court Campbell]

You could have just:

find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | tar -cvf - | ssh hpserv02 "(cd /some/dir; tar xf -)'

[Court Campbell]

You could have just:

find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | tar -cvf - | ssh hpserv02 "(cd /some/dir; tar xf -)"

[Dennis Handly]

>I am combining the tar command and find command as follows. ... arg list too long

You can't really use find and tar for large amounts of files, don't even try. Instead use cpio or pax(1). Both of these take their files from stdin:
find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | \
pax -w -f ./testdev_duts.tar

pax also allows you to to rename the files when you extract so you don't have to cd into the directory root, as JRF suggested.

>JRF: find ... | xargs tar -cvf ./testdev_duts.tar

This will fail since it will overwrite the tarfile over and over.

[James R. Ferguson]

Hi (again):

Ah, oops. Good catch, Dennis. You are correct, the 'find ... | xargs ...' will indeed _overwrite_.

Phil, compare a test case like this where there are a relatively few files:

# find /tmp -type f -mtime -7|xargs tar -cvf /var/tmp/MYTHING

# tar -tvf /tmp/MYTHING

...versus:

# find /tmp -type f -mtime -7|xargs -l2 tar -cvf /var/tmp/MYTHING

# # tar -tvf /tmp/MYTHING

...which takes two (2) arguments at a time. This demonstates the problem Dennis cited.

Regards!

...JRF...

[Phillip Popp]

Your right. It looked like it was tarring it al up, then after it finished it only had the last directory in it. Close but not quite. I have never used the pax command. Is that standard on all hp10.2. also, that command will not put the file to the wrong box, as I have only limited amount of space on it.

Thanks,

phil

[Dennis Handly]

>Is that standard on all 10.20?

If the command is there on one, it should be on all.

>it finished it only had the last directory in it.

I assumed you were happy because JRF's cd approach would reduce the length of the command line.

[James R. Ferguson]

Hi Phil:

Yes, 'pax' exists on 10.20 :

http://docs.hp.com/en/B2355-90128/pax.1.html

Another alternative is to install the GNU tar which allows 'â --files-from=file-nameâ which is designed to read a list of filenames contained in a file as you would generate from a 'find' command.

You can fetch GNU tar from here:

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.20/

...though for 10.20, you will have to compile it yourself

Regards!

...JRF...

[T G Manikandan]

I use like:

tar zcvf test.tar $(find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000)

[Phillip Popp]

Thanks,
The pax command did the trick. did not even know about that command.

Phil

[Steven Schweda]

> I use like:
> [...]

Or, if you'd like a method which _won't_
produce a too-long command line, GNU "tar"
has a "--files-from=file-name"
("-T=file-name") option, and "file-name" can
be "-" (stdin), so you could pipe the "find"
output into it.

[Patrick Wallek]

>>I use like:

>>tar zcvf test.tar $(find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000)

This may be fine on Linux, or with GNU tar, but with the HP-UX native tar the '-z' option (to create a gzip'ed tar file) is not valid.

[T G Manikandan]

Patrick,

Yeah we could do:

tar cvf test.tar $(find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000) && gzip test.tar

[Steven Schweda]

> Yeah we could do:
> [...]

Or, we could learn something, and avoid the
problem of the too-long command line, and
also avoid the temporary file. (HP-UX shells
_do_ know how to deal with pipelines.)