topic Re: tar files greater than size 2 GB in Operating System - HP-UX

tar files greater than size 2 GB

Manjunath Shastry — Mon, 04 Aug 2003 07:01:04 GMT

Greetings Gurus,

I have to send around 40 GB of Data to a client of mine. The data comprises of files larger than 2 GB. I am using HP-UX 11i and my Client is using SGI 's IRIX.

By searching the itrc forums, I found that with a patch applied on HP-UX 11i one can tar files greater than 2 GB.

How about my Client, will he able to restore and untar the files greater than 2 GB ?

What is the best way to go about this ?

Would appreciate your help.

Regards,
Manjunath.

Re: tar files greater than size 2 GB

Robert-Jan Goossens — Mon, 04 Aug 2003 07:04:37 GMT

Hi,

You can use gtar

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.25/

Hope it helps,

Robert-Jan.

Re: tar files greater than size 2 GB

Michael Tully — Mon, 04 Aug 2003 07:11:49 GMT

There are a couple of tools you could use:

GNU tar (which supports files over 2Gb)
dd
dump

Do not try to use the native hpux tar with the patching, as this will only work with another hpux box that is patched the same. It is not compatible with other unix operating systems.

Re: tar files greater than size 2 GB

twang — Mon, 04 Aug 2003 07:25:11 GMT

yes, hpux tar does not support filez over 2G you can try gnu tar
http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.25/

Re: tar files greater than size 2 GB

Alexander M. Ermes — Mon, 04 Aug 2003 07:39:01 GMT

Hi there.
For gzip there is a little workaround.

cat file | gunzip > org_file

Rgds
Alexander M. Ermes

Re: tar files greater than size 2 GB

Charles Harris — Mon, 04 Aug 2003 07:49:11 GMT

Hi, just a quick thought, make sure you have largefile support on you filesystem too, other wise the 2 Gig limit will remain.

cheers,

-ChaZ-

Re: tar files greater than size 2 GB

twang — Mon, 04 Aug 2003 08:05:42 GMT

If you are using 11.11, you may use enhance tar to support the archival of files up to 8GB from the current 2GB, get the patch PHCO_26423:
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHCO_26423&context=hpux:800:11:11

Re: tar files greater than size 2 GB

Robert A. Pierce — Wed, 06 Aug 2003 16:14:15 GMT

Manjunath,

How are you sending the files? Are you sending your client a tape?

If you both have a 40GB tape drive, you can put your files on one tape with e.g.

cd /patches
tar -cvf /dev/rmt/1m ./40gigs_of_files

Then your client can mount the tape and e.g.

mkdir manjunath_patches
cd manjunath_patches
tar -xvf /dev/rmt/1m

. . . and tar will restore the directory "40gigs_of_files" to the client's server.

Hope this helps,

Rob

Re: tar files greater than size 2 GB

Gary Yu — Wed, 06 Aug 2003 16:53:19 GMT

will the 2GB limitation only apply to files on file system or it also apply to tape device, e.g /dev/rmt/0m ?

Gary

Re: tar files greater than size 2 GB

Robert A. Pierce — Wed, 06 Aug 2003 18:02:09 GMT

Gary,

Correct. We use tar to backup a filesystem to DLT, about 20 GB of data.

Works fine, we've even restored files from the tape.

Rob

Re: tar files greater than size 2 GB

Wouter Jagers — Wed, 06 Aug 2003 23:08:20 GMT

The GNU tar definately works. We've run into the same problem a couple of times, and it usually does the job nicely.

When tarring to disk, you would also have to make sure your filesystem is mounted with the largefiles mount option.

regards
Wouter

Re: tar files greater than size 2 GB

Bill Hassell — Thu, 07 Aug 2003 02:36:03 GMT

To answer Gary's question about 2Gb and tape drives, serial devices like a tape have no concept of a file. It is just one long stream of data. tar doesn't know anything about tapes either. It just writes the data until an error occurs pr the end-of-tape status is reported. So you can store millions of terabytes of data on any tape drive without being limited by HP-UX (the tape might not be long enough though, but that isn't an HP-UX problem).

It's important to note that tar (and cpio and similar legacy backup programs) are way out of date for today's backup requirements. HP's patch is only good for 8Gb (not 10Gb or even 100Gb) and as mentioned, there is no similar patch for any other flavor of Unix. Gtar is a good (free) alternative BUT it must be ported to the destination system.

And with that much data, using utilities (like tar) that were designed for 7-track reel-to-reel tapes that stored a whopping 20 megs is probably not the best choice for important information. I would check out commercial solutions that cross multiple platforms. The tools handle errors, recover from danaged tapes, handle multiple reels and auto-loaders, and can backup/restore over the network. Vendors are: HiComp's HiBack, HP's Omniback/DataProtector, Veritas, etc

Re: tar files greater than size 2 GB

Gary Yu — Thu, 07 Aug 2003 12:10:33 GMT

Thanks Bill and Robert, the explanation is very clear.

Gary