topic Re: Tar and find command in Operating System - HP-UX

Tar and find command

Phillip Popp — Thu, 12 Mar 2009 15:50:38 GMT

Hi Running 10.2. I have two machines. I want to tar up files in a rather large directory older then say 3 years. I am combining the tar command and find command as follows. I can not tar the file to machine called hpserv01 as there is not enough space left on the hard drive, so I must build the tar file to the machine called hpserv02. When I try this command from hpserv01 it give me a return message of arg list too long. If I put in more then 1000 to reduce the size of the arguments it tells me o file, no data. Can some one give me some guidence on how to get past the list too long issue.

command from hpserv01: tar -cvf ./testdev_duts.tar `find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000`

any help would be greatly appreciated.

Thanks,

Phil

Re: Tar and find command

James R. Ferguson — Thu, 12 Mar 2009 16:10:48 GMT

Hi Phil:

Try this:

# find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | xargs tar -cvf ./testdev_duts.tar

Regards!

...JRF...

Re: Tar and find command

Phillip Popp — Thu, 12 Mar 2009 16:26:16 GMT

Thanks James. My only problem is that I then want to untar on hpserv02 and using the /net/hpserv01/LTX/hp_testdev_duts/ -mtime +100 will put it back where it came from when I untar. I eventually want to erase those files from hpserv01 when I have the data sucessfully untarred on hpserv02.

I would do a mv command but afraid I will loose my data and screw things up.

Any ideas ?

Thanks,

phil

Re: Tar and find command

James R. Ferguson — Thu, 12 Mar 2009 16:35:14 GMT

Hi (again) Phil:

With 'tar' you should create archives with _relative_ paths instead of absolute ones unless (1) you want to "go nuts"; or (2) use 'pax' to strip the leading '/' to make a relative path.

That said, change you find to output relative paths:

# cd /path
# find . -mtime +1000

Regards!

...JRF...

Re: Tar and find command

Phillip Popp — Thu, 12 Mar 2009 16:50:21 GMT

Sweet!! that did the trick.

Thanks.

Phil

Re: Tar and find command

Court Campbell — Thu, 12 Mar 2009 17:43:42 GMT

You could have just:

find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | tar -cvf - | ssh hpserv02 "(cd /some/dir; tar xf -)'

Re: Tar and find command

Court Campbell — Thu, 12 Mar 2009 17:43:53 GMT

You could have just:

find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | tar -cvf - | ssh hpserv02 "(cd /some/dir; tar xf -)"

Re: Tar and find command

Dennis Handly — Thu, 12 Mar 2009 17:48:40 GMT

>I am combining the tar command and find command as follows. ... arg list too long

You can't really use find and tar for large amounts of files, don't even try. Instead use cpio or pax(1). Both of these take their files from stdin:
find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000 | \
pax -w -f ./testdev_duts.tar

pax also allows you to to rename the files when you extract so you don't have to cd into the directory root, as JRF suggested.

>JRF: find ... | xargs tar -cvf ./testdev_duts.tar

This will fail since it will overwrite the tarfile over and over.

Re: Tar and find command

James R. Ferguson — Thu, 12 Mar 2009 17:56:32 GMT

Hi (again):

Ah, oops. Good catch, Dennis. You are correct, the 'find ... | xargs ...' will indeed _overwrite_.

Phil, compare a test case like this where there are a relatively few files:

# find /tmp -type f -mtime -7|xargs tar -cvf /var/tmp/MYTHING

# tar -tvf /tmp/MYTHING

...versus:

# find /tmp -type f -mtime -7|xargs -l2 tar -cvf /var/tmp/MYTHING

# # tar -tvf /tmp/MYTHING

...which takes two (2) arguments at a time. This demonstates the problem Dennis cited.

Regards!

...JRF...

Re: Tar and find command

Phillip Popp — Thu, 12 Mar 2009 17:56:41 GMT

Your right. It looked like it was tarring it al up, then after it finished it only had the last directory in it. Close but not quite. I have never used the pax command. Is that standard on all hp10.2. also, that command will not put the file to the wrong box, as I have only limited amount of space on it.

Thanks,

phil

Re: Tar and find command

Dennis Handly — Thu, 12 Mar 2009 18:05:27 GMT

>Is that standard on all 10.20?

If the command is there on one, it should be on all.

>it finished it only had the last directory in it.

I assumed you were happy because JRF's cd approach would reduce the length of the command line.

Re: Tar and find command

James R. Ferguson — Thu, 12 Mar 2009 18:08:02 GMT

Hi Phil:

Yes, 'pax' exists on 10.20 :

http://docs.hp.com/en/B2355-90128/pax.1.html

Another alternative is to install the GNU tar which allows 'â --files-from=file-nameâ which is designed to read a list of filenames contained in a file as you would generate from a 'find' command.

You can fetch GNU tar from here:

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.20/

...though for 10.20, you will have to compile it yourself

Regards!

...JRF...

Re: Tar and find command

T G Manikandan — Fri, 13 Mar 2009 11:57:01 GMT

I use like:

tar zcvf test.tar $(find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000)

Re: Tar and find command

Phillip Popp — Fri, 13 Mar 2009 11:59:14 GMT

Thanks,
The pax command did the trick. did not even know about that command.

Phil

Re: Tar and find command

Steven Schweda — Fri, 13 Mar 2009 14:57:31 GMT

> I use like:
> [...]

Or, if you'd like a method which _won't_
produce a too-long command line, GNU "tar"
has a "--files-from=file-name"
("-T=file-name") option, and "file-name" can
be "-" (stdin), so you could pipe the "find"
output into it.

Re: Tar and find command

Patrick Wallek — Fri, 13 Mar 2009 15:10:46 GMT

>>I use like:

>>tar zcvf test.tar $(find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000)

This may be fine on Linux, or with GNU tar, but with the HP-UX native tar the '-z' option (to create a gzip'ed tar file) is not valid.

Re: Tar and find command

T G Manikandan — Fri, 13 Mar 2009 15:46:41 GMT

Patrick,

Yeah we could do:

tar cvf test.tar $(find /net/hpserv02/LTX/hp_testdev_duts/ -mtime +1000) && gzip test.tar

Re: Tar and find command

Steven Schweda — Fri, 13 Mar 2009 17:23:47 GMT

> Yeah we could do:
> [...]

Or, we could learn something, and avoid the
problem of the too-long command line, and
also avoid the temporary file. (HP-UX shells
_do_ know how to deal with pipelines.)