topic Re: Why does tar not work? in Operating System - HP-UX

Why does tar not work?

Thomas Schler_1 — Tue, 06 Apr 2004 10:51:10 GMT

Hi,

I need to create a tar archive using a combination of find, xargs, and tar. E.g.

find tmp | xargs tar fcv tmp.tar

(Assume some more options of find for filtering purposes.)

find produces an output of several hundred paths. All these files shall be put into one tar archive "tmp.tar". But this does not work. Although, I do not get an error message, the resulting tar archive never reflects the paths delivered by the find process.

I tried some other variants:

% tar fc tmp.tar `find tmp`

% tar fc tmp.tar any_file
% find tmp | xargs -n 10 tar fr tmp.tar

% tar fc tmp.tar any_file
% find tmp -exec tar fr tmp.tar {} \;

Nothing works. What is going wrong? What is the solution?

(Is it time to change from HP-UX 11.00 to Linux?)

Re: Why does tar not work?

Patrick Wallek — Tue, 06 Apr 2004 10:57:27 GMT

No, no need to change to Linux. You just need to get your command line options in the correct order.

Your 'f' option should immediately preceed the name of the file you are taring t.

# tar -cvf tmp.tar filenames

So in your examples:

# find tmp | xargs tar cvf tmp.tar -

Note the '-' at the end of the tar command as well. That tells tar to read the file names from standard input.

Re: Why does tar not work?

RAC_1 — Tue, 06 Apr 2004 10:57:35 GMT

Is it find tmp | xargs tar fcv tmp.tar
or find /tmp | xargs tar -fcv tmp.tar

Anil

Re: Why does tar not work?

Sanjay_6 — Tue, 06 Apr 2004 11:04:56 GMT

Hi,

the command works for me. I just ran the command that thomas has mentioned and got a tar file. What is the error you are getting. Hope you are not running out on filesystem space.

Hope this helps.

Regds

Re: Why does tar not work?

Thomas Schler_1 — Tue, 06 Apr 2004 11:09:45 GMT

Patrick,

unfortunately, there's no effect using '-' or using it not. Again, no error message, but:

% find tmp | xargs tar cvf tmp.tar -
% find tmp | wc
336 337 8656
% tar ft tmp.tar | wc
35 35 675

(Using xargs, '-' should be obsolete.)

Something is going wrong here. I assume, it has something to do with LINE_MAX, see xargs(1) and limits(5).

But, what is the work-around?

Re: Why does tar not work?

Thomas Schler_1 — Tue, 06 Apr 2004 11:14:36 GMT

Sanjay,

thank you for your test; no, file space is ok.

Please, see my answer to Patrick. Would you, please, again do the test with some hundreds of files, and, say about, 50 MB?

Re: Why does tar not work?

Marvin Strong — Tue, 06 Apr 2004 11:15:17 GMT

#find dir | xargs tar cvf dirnam.tar -

works for me here, provided I'm in the correct directory.

Re: Why does tar not work?

Thomas Schler_1 — Tue, 06 Apr 2004 11:19:57 GMT

Marvin,

please, do following test:

% find dir | xargs tar cvf dirnam.tar
% find dir | wc
output??

% tar ft dirnam.tar | wc
output??

Re: Why does tar not work?

Jason W. Neiss — Tue, 06 Apr 2004 11:24:00 GMT

> % tar ft dirnam.tar | wc

Again, the 'f' (file) option should come right before the filename:

% tar tf dirnam.tar | wc

Looks to me like this is consistently your issue...

Jason

Re: Why does tar not work?

Marvin Strong — Tue, 06 Apr 2004 11:26:59 GMT

# tar tvf mfstrong.home.tar | wc
79 633 6292
root@hp19rm4 [/tmp]
# find /home/mfstrong | wc
105 106 3789

hmm I see your point.

Re: Why does tar not work?

Thomas Schler_1 — Tue, 06 Apr 2004 11:42:00 GMT

Jason,

in my opinion, there's no problem with the order of 'ft' or 'tf' or such things.

Marvin, Sanya,

anything new??

Patrick,

already at home? Really, no need to change to Linux?

Re: Why does tar not work?

Jeroen Peereboom — Tue, 06 Apr 2004 11:56:51 GMT

L.S.

I think the problem is within the command sequence.

Xargs takes as many arguments as possible, and starts the tar command with all these arguments. This means that tar will be started multiple times, each time overwriting the previously made tarfile. Your tarfile probably only contains the last XX files.

Don't use xargs, just do
find .... | tar cvf tarfile.

JP.

Re: Why does tar not work?

Marvin Strong — Tue, 06 Apr 2004 11:56:56 GMT

don't know whats up with tar, but if your aren't tied to tar, you could use cpio.

find dir | cpio -oc > output

my results:
# find scripts | wc
68 68 1528
root@hp19rm4 [/home/mfstrong]
# find scripts | cpio -oc > scripts.cpio
349 blocks
# cpio -ict < scripts.cpio | wc
349 blocks
68 68 1528

Re: Why does tar not work?

RAC_1 — Tue, 06 Apr 2004 12:05:38 GMT

what shell you are using? Seems csh.
type args, type tar

Anil

Re: Why does tar not work?

Shannon Petry — Tue, 06 Apr 2004 12:10:31 GMT

There can be problems with the order arguments are given to many commands. I do not think that this is the case here, but best practices should be to give "f" as the last argument, and the action first, in this case "c".

The reason for this is that other arguments may require additional arguments. "b" immediately comes to mind.
I.E.
tar -cvbf 2048 -

To fix the problem, try a different approach.

tar -cvf myfiles.tar `find . -depth -print`

Notice the grave marks enclosing the find command. The grave marks tell the shell to execute the command enclosed to get it's arguments.

Regards,
Shannon

Re: Why does tar not work?

Shannon Petry — Tue, 06 Apr 2004 12:30:30 GMT

I just wanted to add a few things after re-reading a bit in the thread.

"find tmp" is not compiant usage of the find command. This could produce an empty list, or pathing issues.

You do not state that an arcive is not made, just that the path is not reflected.

Is an archive made at all? if so what is the contents? There should be something in archive.

If you want absolute pathing in the file, then find needs to give an absolute path.

find /tmp -depth -print
makes the absolute path /tmp

find . -depth -print
makes the cwd the absolute path.

Another problem I noticed is that in your -exec example you do not give the path to tar, which is required for execution.

find /tmp -depth -exec /usr/bin/tar rf myfile.tar {} \;

Try not to be lazy with command arguments. While it may work under certain circumstances, it will not work in all cases.

Lastly, which tar are you using? and are you using a hybred find? It can make a difference if LD_LIBRARY_PATH needs to be adjusted.

Regards,
Shannon

Re: Why does tar not work?

Jeroen Peereboom — Tue, 06 Apr 2004 13:02:16 GMT

I just checked on a Linux system.If you have many files, only the files found 'last' will be in the archive.So you may want to use an 'append' or 'update' mode. Linux tar has an 'r' flag, for append mode.This gave a weird result: the tar arhive contained twice as many files as were found by find. I do not understand (yet).So, using xargs causes the problem.My suggestion to use find ... | tar cvf is wrong. Won't work.JP

Re: Why does tar not work?

Patrick Wallek — Tue, 06 Apr 2004 13:05:47 GMT

Probably the easiest thing to do is:

# cd /tmp
# find . -print > /dir/file_list
# tar -cvf file.tar $(cat /dir/file_list)

Re: Why does tar not work?

john korterman — Tue, 06 Apr 2004 13:06:55 GMT

Hi,
I think I once experienced something similar, namely that the argument list for xargs was too long. I also seem to remember that xargs does only produce an error message if you use it with the -x option. Try testing your find commands like this:
# find | xargs -x 1>/dev/null

If it producces an error message like "xargs: arg list too long" we know what the problem is. If you can provoke that error message then try executing the above command without the "-x" option and without redirection.

regards,
John K.

Re: Why does tar not work?

Jeroen Peereboom — Tue, 06 Apr 2004 13:38:09 GMT

John,

by definition xargs cannot produce an argument list that is too long. Xargs will split the arguments such that the list is not too long, and the command (tar in this case) is called as few times as possible.

My point is: xarsg will call tar a few times because there are so many files. Only the files (arguments) in the last invocation of tar (by xargs) will be in the tar archive. All previously created archives are overwritten.

JP

P.S. Of course, if you add an argument manually, the arglist may become too long.

P.S. 2. The number of arguments in the list is smaller if filenames are longer.