tarring 60GB contents

[Shivkumar]

Hello,

We want to tar 60GB contents. Will traditional tar support allow this or we need to install GNU tar or some other options or tricks ?

Thanks,
Shiv

[Steven Schweda · Accepted solution]

Define "traditional".

As I recall, the file size limit in old "tar"
programs is on the size of a member file (2GB
or 8GB, depending on how traditional), not on
the size of the archive itself.

Old "tar" programs may also have trouble with
long path names (>100 characters), and
probably several kinds of other things.

"pax" is generally better, but I'd probably
choose GNU "tar" unless I had some good
reason not to.

When in doubt, run the experiment.

[Patrick Wallek]

>>Define "traditional".

Would "HP-UX native tar" be a better term?

>>As I recall, the file size limit in old
>>"tar" programs is on the size of a member
>>file (2GBor 8GB, depending on how
>>traditional), not on the size of the
>>archive itself.

Absolutely correct, with a caveat.

If there were a limit on the total size of the archive then you would not be able to use tar to write to high capacity tapes.

The one caveat is if you are creating a tar file on a file system. If you don't have large files enabled then it will error when it reaches 2GB in size. If you do have large files enabled then the tar file can grow until it potentially fills the file system.

>>Old "tar" programs may also have trouble
>>with long path names (>100 characters),
>>and probably several kinds of other things.

I **think** this has been changed. I see nothing in the tar man page about long **path** names.

Restrictions the man page mentions (in a nutshell):
1) When patched, max file size of 8GB for files in the archive
2) UID/GIDs above 2048K not supported
3) Direct quote from man page: "Due to internal limitations in the header structure, not all file names of fewer than 256 characters fit when using the N function modifier. If a file name does not fit, tar prints a message and does not archive the file."
4) Link names limited to 100 characters.

>>"pax" is generally better, but I'd
>>probably choose GNU "tar" unless I had
>>some good reason not to.

That could work, but then you have to make sure you have that on your destination system as well.

If there are restrictions / prohibitions about what you can or can't install, then this could be problematic. Some places still frown on using open source software in production environments.

>>When in doubt, run the experiment.

Definitely.

[Shivkumar]

Looks like on my servers both traditional and GNU tar installed. How do i know which is GNU tar ?

[Steven Schweda]

> [...] installed.

Installed where? I'd expect to find GNU
"tar" in "/usr/local/...". Around here,
"gtar" is a link to GNU "tar", so I can
always see which one I'm using.

> How do i know which is GNU tar ?

Ask it?

/path/tar --version

[Steven Schweda]

> Would "HP-UX native tar" be a better term?

Perhaps, if combined with an HP-UX version
and/or patch info.

> If there are restrictions / prohibitions
> about what you can or can't install, then
> this could be problematic.

Lame supported software can cause problems,
too. Everything's complicated.

My knowledge of "tar" comes mostly from some
work I did on VMSTAR to add some of the GNU
"tar" extensions (and symbolic links). When
I got to the part where the file size was
stored as eleven octal digits, I almost
spewed my beverage. That and the
hundred-character name limit. Yow. The GNU
extensions to work around the original design
(cough) limitations without creating a whole
new format required some actual cleverness.