topic Re: backup using tar in Operating System - HP-UX

backup using tar

Edwin Ruiz_2 — Thu, 20 Jan 2005 09:59:51 GMT

Hi:

I need to backup one file of 60 GB using tar becuase i need to restore under Linux and Solaris but i get that messages when i try that
# tar -cvf /dev/rmt/0mn fullVENTASWedJan19.dmp
tar: Size of fullVENTASWedJan19.dmp > 2GB. Not dumped.

is there some patch for correct that??

Re: backup using tar

Geoff Wild — Thu, 20 Jan 2005 10:05:01 GMT

You have to use GNU tar - The size of files you put into a tar archive (not the archive itself) is limited to 11 octal digits, the max. size of a single file is thus
ca. 68 GBytes.

HP Tar without patches is 2GB, with - is 8GB....

Rgds...Geoff

Re: backup using tar

twang — Thu, 20 Jan 2005 10:09:55 GMT

You can get it here:

http://hpux.cs.utah.edu/hppd/hpux/Gnu/tar-1.13.25/

Re: backup using tar

Peter Godron — Thu, 20 Jan 2005 10:10:00 GMT

Edwin,
you can also compress the files first, which will reduce the backup space requirements.Of course at the cost of having to run un/compress!

Re: backup using tar

Patrick Wallek — Thu, 20 Jan 2005 10:11:28 GMT

60GB with the native tar command absolutely will not work. YOu must remember that tar is an OLD unix utility that was not designed to work with the large files we see today. The unpatched version on HP-UX can only archive files up to 2GB. Even the patched version can only go up to 8GB.

I don't know if Sun and Linux have the same limitations or not.

You could try GNU tar, but you'll have to make sure that you install the appropriate version of GNU tar on the Sun and Linux machines as well.

Re: backup using tar

Edwin Ruiz_2 — Thu, 20 Jan 2005 10:15:24 GMT

thaks for ur recomendations

Re: backup using tar

Stephen Keane — Thu, 20 Jan 2005 10:17:45 GMT

Or you could use the split command (-b flag) and split the file into multiple files that would fit under the tar filesize limit. Of course you would need a lot of free space to do that, on each of the servers (source and destination). Then you could recombine the bits on the target servers.

Re: backup using tar

twang — Thu, 20 Jan 2005 10:18:12 GMT

Add to above, Patrick is right, we have to patch the 11i tar, it can support backing up 8GB file, to obtain the patch here:
http://www2.itrc.hp.com/service/cki/patchDocDisplay.do?patchId=PHCO_28992

Re: backup using tar

Bill Hassell — Thu, 20 Jan 2005 10:30:24 GMT

60 Gb is definitely not a job for tar. Even if you get the large-file version of GNUtar loaded on HP-UX, Linux and Solaris, this is a utility that has virtually no error handling or error recovery and the amount of data is so large that checking it for accuracy is almost impossible (ie, checksums) so you have to rely on the reliability of the tape drives. If this is to be a regular production task, get a commercial quality backup program that will add additional checking and error recovery features.

Re: backup using tar

Chris Vail — Thu, 20 Jan 2005 14:07:49 GMT

I agree with Bill Hassel that you should get a commercial, 3rd party backup utility to handle files of this size. The one we use is Netbackup, which is pricey.

I note that Netbackup is basically a superset of tar. It breaks up your files into 2GB chunks and backs those up to tape, but does indexing and data validation so that the data is recoverable. It can be configured so that these 2GB chunks each go on different devices, which really cuts down backup and restore times. Its better to have 30 2GB files than 1 60GB file on tape.

I'm a fan of tar, but don't think it appropriate for really large data files.

Chris

Re: backup using tar

TwoProc — Thu, 20 Jan 2005 17:52:19 GMT

Well tar's not going to do it. But, you can sling it over directly via a remsh or ssh command.

First set up remsh and ssh to work across the two systems ...

Second, go to your destination machine .... ( I prefer destination)...

A) Remote Shell (rsh on your Linux or Sun box, remsh on HPUX)
Login into Linux box...
cd /mydest
rsh hpuxmachine "cd /mysource; dd if=./fullVENTASWedJan19.dmp bs=64k" | dd of=./fullVENTASWedJan19.dmp bs=64k &

Now, just wait till it's over...

If data dump won't do it - you can try the "cat" command.

ssh hpuxmachine " cd /mysource; cat fullVENTASWedJan19.dmp " > fullVENTASWEDJAN19.dmp

Well, I've never had a problem with using "cat" instead of dd - but I'm just a bit skeptical ... you could easily download a checksum program from the HPUX porting archive (search google for HPUX porting archive), and do a checksum on both sides and make sure that they match.

As seen above - if you're using ssh(recommended) instead of rsh - just put ssh instead of rsh in the command structure...

Lastly, if this is a one-time-thing - then I'd be really tempted to use dd, tar, cpio, or whatever I can get my hands on... but if this is a recurring thing I'd get some professional backup software. Since others are throwing in their votes - I'd suggest HP Data Protector - I really think it's great. But then again - I'm a big fan of HP products and services... (that could/should be read as throw away your Sun box).

Re: backup using tar

Bill Hassell — Fri, 21 Jan 2005 09:33:40 GMT

Just a note about network backups: LAN backups don't come close to the speed needed to keep modern taoe drives busy. A 100Mbit link averages 3-5Mbytes/sec, way too slow for DDS4 or Ultrium drives. This will cause excessive tape repositioning and delay the backup completion by as much as 200%-400%. Commercial products such as Data Protector can take multiple data streams from several sources to keep the tape drives busy.