writing a large filesystem to tape with tar

[Damien Lass]

I have a 6 gig directory that I need to copy to a 4 gig DDS tape (/dev/rmt/0m). What is the best way to get this directory compressed and written to tape? I do not want to rely soly on hardware compression for this archive. The data written to the tape must be portable as I am going from a HP-UX 11.0 box to a Sun Solaris OS.

Would the following work?
1. tar cvf - ./ | gzip - > /dev/rmt/0m

I am under the belief that a tar will not work with anything larger than 2GIG. If this is the case, what are my options?

All help would be greatly appreciated. Thanks in advance!

[Patrick Wallek]

First question -- What type of data is in the directory?

Depending on the data type, if it is very compressable data, then you may actually be able to get all of the data on the tape with just a generic tar command (tar -cf /dev/rmt/0m ./). This would be the first thing that I would try before getting too fancy with other methods.

Tar will behave very well in this situation. If you were doing something like 'tar -cvf /dir/file.tar ./' and you went over 2GB on the file.tar then you would have problems. The other problem you could have is if any file in the directory you are writing to tape is larger than 2GB.

If neither of those is an issue, then you won't have a problem.

[John Poff · Accepted solution]

Hello,

In HP-UX, tar and cpio have a 2 Gb limit on the file size. The way you have it on your command line, tar should be fine as long you don't have any files in your filesystem that are over 2 Gb. Your way will have tar create a regular archive, and then you are compressing the archive.

I don't know if gzip will work like that or not, but I've had pretty good luck with the 'compress' program. I'm also a fan of cpio, so I'd try it like this:

find . | cpio -ocm | compress >/dev/rmt/0m

and then on the Sun box:

compress
assuming that those options for cpio are correct for Sun Solaris. I'm not a Sun guy [I don't have any to play with :( ] so I don't know.

I'd suggest trying it with a couple of small files first if you can, that way you will know if it works or not.

JP

[S.K. Chan]

As other have mentioned, the limit is on a single file which is > 2GB. Find out if you have files bigger than 2GB ..
# find / -type f -size +2000000c -exec ls -al {} \;
Those files (if any) can be compressed first before they are tar'ed. The safest (with cross platform tar) is to use just the simplest means ..
# tar cvf /dev/rmt/0m ./
Another concern I have is how would you know if it'll fit the tape (even with gzip). I would almost wanted to suggest splitting the copy to 2 tapes instead, 3GB in each to play safe (at least that's what I would do if I were you).
my $0.02

[T G Manikandan]

check for the
compressdir.

This should bring the file system to the size what you are expecting.
After restoration you should do a uncompressdir
to remove them from .Z extensions.
Just check it out

[Brian M Rawlings]

If I may suggest... unless you are really stuck on using HP's supported utilities (which won't work for you without significant extra efforts on your part, and which may not work even then), have you considered using GNU's tar utility? It doesn't have the 2GB file restriction, and adds a lot of value. It is available for HP-UX from the porting center (centre?), at:
http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.25/

Details on using gnutar/gtar are at:
http://www.gnu.org/manual/tar/html_mono/tar.html

It would involve installing on your HP box, and possibly on the Sun box, but it would do what you are trying to do without any fuss at all, provided that it will fit on the tape.

Speaking of fitting it on the tape, relying on the DAT's HW compression is probably your best choice for compatiblity. The compression/decompression is all hidden from the OS, and all DAT drives that can read a given tape can also uncompress it if it is compressed. One thing you should avoid is multiple compression algorithms. If you compress a file, then try to compress it again, you often end up actually growing the 2-nth version a little. Pick one compression technique and use it once. As I said, I'd just let the tape drive do that part, but lots of ways will work.

Regards, --bmr

[Tom Danzig]

Additionally, the gnu version of tar supports compression as in:

tar -zcvf ....

The z signifies compression.