fbackup very very slow

[patrick coutinho]

Hi,

We have an rp7410 with 4 cpu's and 12 GB memory and 20 gb swap. we are backing up a lun which is 102 gb in size and is on an EVA 5000 with 1 gb HBA.

This backup is taking far too long. Index is activated. this backup is going on for 8 hours now. total no. of files to be backed up is 678,000 and there is a deep directory structure. Please help.

Thanks in advance,

Rgds

Pat

[A. Clay Stephenson]

Well, without knowing what you are backing up to, it's rather difficult to be helpful. If you are backing up to an AIT or DLT device then you should increase readerprocesses to 6 (the maximum), increase blocksperrecord and records.

You could also be trying to backup files which are being updated so that retries are attempted. In that case, reduce maxretries.

You might consider loading the Trial version of DataProtector for better performance.

[Steven E. Protter]

This could be almost anything. It could be bad termination, bad scsi cable, even a fault wwith the or the drive itself.

It could be a problem with the fiber card or LUN as well. You might want to check with SAN administration to see if there are issues there.

Lets start with the ioscan command and look for problems. The number of files or depth of the directory structure should not be that big a problem. I've backed up bigger and deeper with Ultrium drives. I've done it in substantially less time than 8 hours too.

SEP

[Bill Hassell · Accepted solution]

DO NOT run fbackup without a config file. The defaults are designed for (very old) 1/2" reel-to-reel tape drives. Use this file with -c option:

blocksperrecord 256
records 32
checkpointfreq 1024
readerprocesses 6
maxretries 5
retrylimit 5000000
maxvoluses 200
filesperfsm 2000

This should significantly reduce the backup time. Make sure your tape drive is NOT connected to the same interface as your disks.

[patrick coutinho]

Thanks very much guys,

We are using DLT IV with compression thru sam. The backup is COLD. Thanks Bill. I will try out the config file. What does filesperfsm mean ? We have tried this backup twice now and had to kill it because of the slowness. How long in your estimation should a 102 GB backup take with the tape drive locally connected. Its a DLT 8000 drive.

Thanks once again,

Patrick

N.B. Points allocated.

[A. Clay Stephenson]

filespersm is the number of file between fast search marks. A smaller numer allows faster restores of individual files at the expense of a little overhead.

The main criteria for getting good performance out of a DLT is to keep the drive streaming. Is the drive writing almost continuously or does it write for a second or two then pause and write again briefly. If the drive is stopping and starting the throughput is greatly reduced from nominal.
Realistic throughputs for this drive are about 40-50 GB/hr although the specifications are much higher. If this is copper SCSI, the DLT drive must be on a dedicated bus.

Typical schemes for backing up large amounts of data use some sort of snapshotting or mirror spliting. The snapshots take only a few seconds. You can then restart the application and back up at leisure using the snapshots. You then don't really care how long the backups take.

[A. Clay Stephenson]

One final point, all of the fbackup config file paramaters are discussed rather well in the fbackup man page. Man fbackup for a full description.

[patrick coutinho]

Thanks a lot Clay,

We have solved out problem out here. For everyone's benefit what we have done is

- Use a SDLT drive which is located on our Legato backup solution. This drive is connected via fibre to our EVA and thru to the host via HBA. And after using Bill's config file, a test backup just flew past. 64,000 file totalling upto approx 3 GB completed in "verbose" fbackup mode in just 306 seconds.

Thanks everyone.

Rgds

Pat

[Bill Hassell]

filespersm is a highspeed search mechanism. For DDS drives, this is a special marker that can be seen while running the tape about 100x normal speed. Think of it as a big freeway sign. fbackup's index knows how many setmarks must be passed before a particular file will be close. So a file at the end of the tape takes just a few minutes to restore. On DLT and Ultrium drives, setmarks do not exist so tape marks (aka, file separators) are used for the same task.

As mentioned, high performance taoe drives must be kept busy or they will reposition, causing several seconds in lost time (and extra wear on the drive). If you can't keep the tape drive busy, performance will drop to 1/50th to 1/500th normal speed depending on how slow the data arrives. fbackup maximizes back speeds by queueing up several reader processes to open files ahead of the tape, then dumping the data into a shared memory area where it is written to the tape. filespersm is essentially overhead so setting it to 1 will cause 678,000 setmarks to be written and since each setmark occupies a space where megs of data could be written, the overhead will cause the tape to not reach maximum capacity by a significant amount. Set filespersm to 20000 and now the overhead will be very low but recovering a file will jump to a dozen or two minutes because the index can only get to a 20,000 file block.

[patrick coutinho]

Thanks a lot Bill. Sorry for the delay in replying. Just got caught up in work.

Rgds

Pat