Just a note on classic Unix utilities such as tar, cpio, pax, dump, etc. These utilties were designed when massive disk farms contained as much as 100 Mbytes of data (not Gigabytes) and when 32 bits was more than anyone would ever need for addressing. After all, 16 megs of core memory cost $4,000 back in the early 70's.
The standards for tar, cpio, etc are cast in concrete by the original design. There simply is no room in the headers to represent a file size larger than 2 Gb. While GNU tools will handle the larger files, these tapes are not interchangeable with other systems unless the same GNU tools are loaded. Largefiles are becoming quite common and some of these tools may be silently truncating the data...not good.
And with filesystems measured in 10's to 1000's of gigabytes, these antiquated tools are very inefficient and unreliable for anything but small file interchange. The inefficiency is seen in the lack of a central directory and no method to quickly position a tape to the desired location. For a 2 Gbyte backup, this is not much of a problem but on average, it will take 1/2 the time of the original backup to retrieve a file from the tape as the tape must be serially read.
Modern backup tools like fbackup or Omniback will place high speed search marks into the data stream every few hundred files. Then in the index file at the beginning of the tape, each file's position is recorded relative to the nearest leading search mark. Thus, frecover can restore a single file in a few minutes even if fbackup required two hours to reach the end of the tape.
Since the majority of reasons to run a file recovery process is due to user mistakes, often just one file or directory, waiting for hours for a restore to complete is not useful.
Also, fbackup understands multi-tape backups and can move a tape changer using the mc command in a chgvol script. This means that finding a file buried in a 5-tape backupp set is as simple as putting in the *last* tape, running frecover and folowing the instructions on which tape to insert for the actual recovery. Similar capability is builtin (no script needed) for Omniback.
Reliability is a very big concern today. With backups measured in dozens of gigabytes, if the backup does not work or the tape has a bad spot, tar/cpio/etc are useless. They have virtually no error recovery code built into the programs and thus, a single bad spot on the tape can leave the rest of the backup useless. Only very sophisticated )and expensive) data recovery services can salvage the tape.
Commercial tools like Omniback have several redundancies built-in along with I/O error recovery that will do the best it can with a bad tape. And these tools support multiple media paths for higher performance (ie, it will use 5 tape drives in parallel for 5x the performance).
Bill Hassell, sysadmin
