Throughput to DDS5 tape drive

[ITeam]

Hi.

I have a customer with the following Server spec:

HP 9000 RP4440 running HPUX 11i
4Gb RAM
Tape device: Internal DDS5/DAT72
Most recent o/s patches applied: Jun 04

This customer runs a nightly data backup which usually writes about 80Gb to tape. When this runs successfully the transfer rate is approx 2Gb/min. It is now a regular occurrence for this to slow down to 200Mb/min & this has a knock-on affect on other processes that cannot run due to the backup still being in progress. If a reboot is performed, the subsequent nightly backup will write to tape at the correct speed. Usually the next night the problem will reoccur though. There are no other processes that should be interferring with this whilst the backup is running.

I have been told by an HP engineer that it is not likely to be a problem with the tape device nor any other hardware. Advised it is possible that software may be responsible. Possibly a memory leak?

Has anyone any ideas or come across this before?

Many thanks,

David Stedman

[harry d brown jr]

IT is NOT a tape drive issue, like the HP engineer pointed out.

You need to look at the software you are using to perform backups (which you didn't post).

Additionally, you NEED to bring your machine up to date with at least the DECEMBER 04 patch bundle!! http://www1.itrc.hp.com/service/patch/releasePage.do?BC=patch.breadcrumb.main|patch.breadcrumb.bundle|patch.breadcrumb.releaseIndexPage|&releaseId=5013-3299


live free or die
harry d brown jr

[Steven E. Protter]

You will never get the maximum throughput on this tape drive. Your milage (performance) may vary based on what software you are using.

Hardware concerns:

Don't put the tape on a daisy chain with disk or other high volume scsi devices. This will significantly hinder performance.

Otherwise get decent software, keep the machine patched as Harry says and have a good time.

SEP

[Bill Hassell]

As mentioned, this is a backup software issue, not a hardware problem. As long as the tape drive has a dedicated interface card (not on the same chain as disks), then performance will be dictated by the speed in which the data can be sent to the tape. tar, cpio, dump, pax, etc are all legacy tools deisgned for 1/2" reel-to-reel tape drives and have no provision to open multiple files and keep the tape drive as busy as possible.

Modern tape drives (DDS-5, Ultrium, etc) have enormous throughput but are crippled by simple file backup tools. The reason is that something like tar will:

find a file
read the file one record at a time
write each record to the tape
close the file

And if anything else is going on in your system, especially in the same filesystem that is getting backed up, tar has to stand in line with all the other requests. There is no way to improve tar (and tar/cpio/etc should never be used with large files and important databases).

It's not just the slower response for file access..the 1/10th normal performance is also caused by missing the buffer requirements in the tape drive. Modern tapes are streamers, that is, they do not write one record at a time. Rather, they start up and continuously record non-stop. If data isn't supplied fast enough, the tape drive slows down, stops, backs up a ways, starts up, syncs to the previous records and then starts writing at the next record. This restart can take 3-8 seconds, an enormous amount of time, especially if it occurs dozens of times each minute. And thus, the very slow throughput.

So if the hardware is dedicated, start by using fbackup rather than legacy tools. Make sure you *ALWAYS* specify the following config file (fbackup defaults are for very slow systems and reel-to-reel drives):

blocksperrecord 512
records 64
checkpointfreq 1024
readerprocesses 6
maxretries 5
retrylimit 5000000
maxvoluses 200
filesperfsm 2000

fbackup will spawn 6 separate processes to keep the tape drive busy. Other commerical backup software products for HP-UX will do similar things to maximize the throughput. You'll always get the best performance when the system is not busy.

[ITeam]

Thanks for the helpful replies.

Just one additional thing. I forgot to mention that the problem occurs after a tape is NOT inserted into the drive. For eg, someone forgets to insert a tape & as a result, the backup fails. Then the following night the backup throughput issue starts to occur.

Not sure if this changes anything.

Thanks.

[Bill Hassell]

Sounds like the backup script needs some work. I would never set up a backup script that did not check for a tape, that it is write enabled and that it is the expected tape to use. Look at the mt command, specifically the status option.