tar files greater than size 2 GB

[Manjunath Shastry]

Greetings Gurus,

I have to send around 40 GB of Data to a client of mine. The data comprises of files larger than 2 GB. I am using HP-UX 11i and my Client is using SGI 's IRIX.

By searching the itrc forums, I found that with a patch applied on HP-UX 11i one can tar files greater than 2 GB.

How about my Client, will he able to restore and untar the files greater than 2 GB ?

What is the best way to go about this ?

Would appreciate your help.

Regards,
Manjunath.

[Robert-Jan Goossens]

Hi,

You can use gtar

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.25/

Hope it helps,

Robert-Jan.

[Michael Tully]

There are a couple of tools you could use:

GNU tar (which supports files over 2Gb)
dd
dump

Do not try to use the native hpux tar with the patching, as this will only work with another hpux box that is patched the same. It is not compatible with other unix operating systems.

[twang]

yes, hpux tar does not support filez over 2G you can try gnu tar
http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.25/

[Alexander M. Ermes]

Hi there.
For gzip there is a little workaround.

cat file | gunzip > org_file

Rgds
Alexander M. Ermes

[Charles Harris]

Hi, just a quick thought, make sure you have largefile support on you filesystem too, other wise the 2 Gig limit will remain.


cheers,

-ChaZ-

[twang]

If you are using 11.11, you may use enhance tar to support the archival of files up to 8GB from the current 2GB, get the patch PHCO_26423:
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHCO_26423&context=hpux:800:11:11

[Robert A. Pierce]

Manjunath,

How are you sending the files? Are you sending your client a tape?

If you both have a 40GB tape drive, you can put your files on one tape with e.g.

cd /patches
tar -cvf /dev/rmt/1m ./40gigs_of_files

Then your client can mount the tape and e.g.

mkdir manjunath_patches
cd manjunath_patches
tar -xvf /dev/rmt/1m

. . . and tar will restore the directory "40gigs_of_files" to the client's server.

Hope this helps,

Rob

[Gary Yu]

will the 2GB limitation only apply to files on file system or it also apply to tape device, e.g /dev/rmt/0m ?

Gary

[Robert A. Pierce]

Gary,

Correct. We use tar to backup a filesystem to DLT, about 20 GB of data.

Works fine, we've even restored files from the tape.

Rob

[Wouter Jagers]

The GNU tar definately works. We've run into the same problem a couple of times, and it usually does the job nicely.

When tarring to disk, you would also have to make sure your filesystem is mounted with the largefiles mount option.

regards
Wouter

[Bill Hassell]

To answer Gary's question about 2Gb and tape drives, serial devices like a tape have no concept of a file. It is just one long stream of data. tar doesn't know anything about tapes either. It just writes the data until an error occurs pr the end-of-tape status is reported. So you can store millions of terabytes of data on any tape drive without being limited by HP-UX (the tape might not be long enough though, but that isn't an HP-UX problem).

It's important to note that tar (and cpio and similar legacy backup programs) are way out of date for today's backup requirements. HP's patch is only good for 8Gb (not 10Gb or even 100Gb) and as mentioned, there is no similar patch for any other flavor of Unix. Gtar is a good (free) alternative BUT it must be ported to the destination system.

And with that much data, using utilities (like tar) that were designed for 7-track reel-to-reel tapes that stored a whopping 20 megs is probably not the best choice for important information. I would check out commercial solutions that cross multiple platforms. The tools handle errors, recover from danaged tapes, handle multiple reels and auto-loaders, and can backup/restore over the network. Vendors are: HiComp's HiBack, HP's Omniback/DataProtector, Veritas, etc

[Gary Yu]

Thanks Bill and Robert, the explanation is very clear.

Gary