Backup takes to long (9 hour)

[Piet Timmers]

Hi,

We have a 36GB disk, and it takes about 9 hour to backup.

$ dir disk14:[000000...]/gr

Grand total of 3560 directories, 6117595 files.

I know this is a lot, but is there a way to backup this disk in a shorter time. HP is working on it, but mayby some of you have good ideas.

Greetings,

Piet Timmers

[labadie_1]

You have not said if this backup is disk to tape or to disk ?

it would help to have the exact backup command, and to check if you have a /block=...

qualifier. if you do not have, the default of 8192 gives the worst results :-)

put 32768 or 65536

use an account with correct quotas for backup, and read the doc (on this mirrot)
http://www.pi-net.dyndns.org/docs/openvms0731/731final/6017/6017pro_046.html

search for

11.7 Setting Process Quotas for Efficient Backups

If you backup to tape, please post a
$ sh dev tape /full
and we should be able to tell you what backup time should be reasonable.

Regards

Gerard

[Jan van den Ende]

Mogge Piet,

Backup is a greedy beast, if you are willing to give it 'room to move'.
For performance:
Specify as big as possible a blocksize
(32 or 64 K, that's 32767 or 65535).
Especially for disks with many small fragments (fragmented files and/or small files) BACKUP uses memory as much as possible for optimisition:
First it allocates as much as possible of workspace. Then it processes header info in such a way that it (only) maps the file(segments) to that workspace IN A CONTIGUOUS WAY. When the space is so full that the next segment would overflow it, then it issues IO requests for ALL mapped segments at once.
This forces the disk into a sequential sweep, delivering the required diskblocks as it passes them. This very much reduces head movements, which are relatively very time consuming. If all segments (of this charge) are read in, then essentially ONE IO to move a contiguous piece of memory to tape is given, and the loop re-iterates.
This shows WHY an as big as possible WSQUOTA for the account doing the backup is very benificial. BUT, if WSEXTENT is bigger, you might well allocate MORE, and have to find part of it in the PAGEFILE... you do'nt want that. So: WSQUO & WSEXT should be equal.

You should be wise to specify a dedicated account, ONLY for backups.

If you already have all these measures in place, then it looks like you can only be helped with faster devices, but considering, 6M files usually takes us some 1.5 hours.

Tot ziens maar weer

jpe

[Piet Timmers]

Thanks Gerard and Jan,

It is a backup from disk to tape and all the settings are as optimal as possible, user account is tuned, blocksize=65535 and so on. By the way we use ABS to make this backup. According to the HP specialist a rollback should improve because we are talking over a disk with a lot of small file and resonable fragmentation. See below:

DISK$DATADISK1401 2-FEB-2004 06:00:24.29

The fragmentation index is 26.0
1 - 20.9 is excellent
21 - 40.9 is good
41 - 60.9 is fair
61 - 80.9 is poor
81 - 100 indicates a badly fragmented disk
Approximately 6.0 (out of 80.0 possible) is due to file fragmentation
Approximately 20.0 (out of 20.0 possible) is due to freespace fragmentation


Freespace Summary:
Total free space: 24495258 blocks
Percentage free: 34 (rounded)
Total free extents: 997920
Maximum free extent: 284232 blocks, LBN: 35273079
Minimum free extent: 3 blocks, LBN: 26140362
Average free extent: 24 blocks
Median free extent: 27 blocks


File Fragmentation Summary:
Number of files (with some allocation): 5846744
Total file extents on the disk: 6061141
Average number of file extents per file: 1.036669
Median number of file extents per file: 1

Most Fragmented File:
[BPS.UTR.DOC_HS.02725]02724608002.A03;1 (7 extents)

There were no files with 8 or more extents

[Jan van den Ende]

Piet,

File fragmenting 1.03 is nearly perfect!
Backup is only interested in allocated segments, and absolutely NOT in free space fragmentation, so a restore is just pity about the work invested.
Stupid question: we ARE talking about /IMAGE backup, are we?
Like Gerard asked: post
the exact backup command.
What tape drive you have you.
The disk type.
How is your disk connected?

jpe

[Piet Timmers]

Here we go:

We use ABS for backup and ABS makes the following backup command:

19:19:29 THREAD #13: $ BACKUP /IMAGE DISK14: -
19:19:29 THREAD #13: _$ /LIST=_MBA2433:/FULL -
19:19:29 THREAD #13: _$ -
19:19:29 THREAD #13: _$ /IGNORE=(INTERLOCK) -
19:19:29 THREAD #13: _$ /NOCRC/NOVERIFY -
19:19:29 THREAD #13: _$ 'SHELVE_QUAL''ALIAS_QUAL' -
19:19:29 THREAD #13: _$ /BLOCKSIZE=65535 -
19:19:29 THREAD #13: _$ $2$MGA0:14MAR20041919240./SAVE -
19:19:29 THREAD #13: _$ /EXACT_ORDER -
19:19:29 THREAD #13: _$ /STOR=V2SLS/NOASSIST

Device info:

Disk DSA1401:, device type Generic SCSI disk, is online, mounted, file-oriented
device, shareable, available to cluster, error logging is enabled, device
supports bitmaps (no bitmaps active).

Error count 0 Operations completed 148214805
Owner process "" Owner UIC [SYSTEM]
Owner process ID 00000000 Dev Prot S:RWPL,O:RWPL,G:R,W
Reference count 23 Default buffer size 512
Total blocks 71114623 Sectors per track 254
Total cylinders 13999 Tracks per cylinder 20

Volume label "DATADISK1401" Relative volume number 0
Cluster size 3 Transaction count 23
Free blocks 22900032 Maximum files allowed 16711679
Extend quantity 5 Mount count 2
Mount status System Cache name "_DSA0:XQPCACHE"
Extent cache size 64 Maximum blocks in extent cache 2290003
File ID cache size 64 Blocks currently in extent cache 30
Quota cache size 0 Maximum buffers in FCP cache 10001
Volume owner UIC [1,1] Vol Prot S:RWCD,O:RWCD,G:RWCD,W:RWCD

Volume Status: ODS-2, subject to mount verification, write-back caching
enabled.
Volume is also mounted on UTRN02.

Disk $1$DGA300:, device type HSG80, is online, device has multiple I/O paths,
member of shadow set DSA1401:, error logging is enabled.

Tapeunit info:

Magtape UTRN01$MKA600:, device type COMPAQ SuperDLT1, is online, file-oriented
device, available to cluster, error logging is enabled, controller supports
compaction (compaction disabled), device supports fastskip.

Error count 0 Operations completed 0
Owner process "" Owner UIC [SYSTEM]
Owner process ID 00000000 Dev Prot S:RWPL,O:RWPL,G:R,W
Reference count 0 Default buffer size 2048
Density default Format Normal-11

Volume status: no-unload on dismount, position lost, odd parity.

Succes,

Piet

[labadie_1]

a monitor disk/item=queue during the backup would be interesting: do you have a big queue length ?

[Wim Van den Wyngaert]

Try "mon file" during the operation.
Some cache may need to be enlarged.
(I seem to remember something going slow because the file header cache rate was almost 0)

[Jan van den Ende]

Piet,

For the tape, I noticed "Compaction disabled".
I also did not find /MEDIA=COMPACT in your backup command.

During Backup, compaction can be quite helpful: your tape does about the same in physical writes, but those can contain (depending, but sometimes much) more data.

fwiw,

jpe

[Mohamed K Ahmed]

Piet,

Can you check the cables, terminator and the KZPCA card connecting the system to the tape drive. If using the wrong tape, it might be carrying a very low rate of transfer in contrast to the right tape for connection.


mohamed

[Ian Miller.]

Could either be a hardware problem - check error log etc for disk and tape, or something else running at same time as backup - check what else is running.

[Antoniov.]

Hello Piet,
May be your system works fine. Reading your post I understand you have to backup 46'214'591 block or 23'661'870'592 bytes (about 24Gbs, 2/3 of your HD).
By your posts I've seen you use /VERIFY and you don't use /COMPACT qualifiers; this means you tape work about at 5 Gb per minute and may be a good performance (al least for me and my customer ;-) ).

@Antoniov

[Uwe Zessin]

Hello, Antoniov,
somehow I get different results than you - can you, please, verify my calculations?

According to Piet's post from '11:42:58 GMT' it looks to me like he runs without /VERIFY:
> 19:19:29 THREAD #13: _$ /NOCRC/NOVERIFY -

I agree that he as about 24 GBytes to backup, but I get completely different results for the throughput:

24 GBytes / 9 hours = 2.66 GBytes/hour

2.66 GBytes/hour = 2660 MBytes/hour (2660/60) -> 44.3 MBytes / minute

44.3 MBytes/minute = 44300 KBytes/minute (44300/60) = 738.3 KBytes/second


I wonder what tape media is being used...
Piet ?

[Martin P.J. Zinser]

Hello Piet,

44 MB/min is not really that bad. What else do you have on the first SCSI bus (M/DKAxxx). Uwe will most probably know more about this, but I do seem to remember that the overall bus speed is limited by the slowest device on the bus. If you do have a slow CD-Rom on the bus this might slow down your SDLT. Also is the tape "streaming", i.e. does it get data fast enough or is it repeatedly starting and stopping?

Greetings, Martin

[Jan van den Ende]

Yeah!

Might very well be that Martin has got hold of the tiger's tail: that would definitively be a reason. (Kick myself in the ass for not remembering this one, though it was some years back). Check ( & doublecheck ) before any other effort!

jpe

[Uwe Zessin]

Ah, a common 'myth', which has some truth in it...

Usually, in SCSI, the initiator (HBA) can individually negotiate the transfer rate with each target (disk, tape, CD-ROM, ...), so that each data transfer runs at 'optimal speed'. However, a slow device ties up the bus for a longer time for the same amount of data.

Also, as I have been told, some HBAs from the HP pre-merger side do indeed slow down for all device on the bus.

[Antoniov.]

Sorry,
wrong calculation.
Uwe, you evaluate right values.
My intent was focus on SCSI throughtput and Martine has locate the problem.

@Antoniov

[Wim Van den Wyngaert]

Piet,

May be a silly question. Was the tape initialized on the device itself ? We do an initialize /dens=dlt8000 for every tape we use to make sure that the re-used tape gets its optimal density.

Per default, backup and init re-use the tapes density. So if you have a tape initialized on an older drive you get a lower speed (and capacity).

Groetjes

Wim

[Piet Timmers]

We let do ABS the whole tape handling, so we do not implicit init the tapes.
Todat i will init the tapes by hand and see whether there is a difference.

Piet

[Piet Timmers]

Wim,

$ init/density=dlt8000 gives:
%DCL-W-IVKEYW, unrecognized keyword - check validity and spelling
\DLT8000\

We are using OpenVMS V7.2-2

Greetings,

Piet

[Jan van den Ende]

Piet,


init/media=compact will have the drive use it's compactation info. It should do the job (always has for me).

[Wim Van den Wyngaert]

Jan,

I think you are wrong on this one. We had the problem that the tapes were initialized on TK88 (or whatever). The init/compact still keeps the TK88 setting. You have to reinit the tape with the highest density the drive supports.

Piet,

Try to use a new tape. After the init the tape will be in the highest density the drives supports. Also make sure that the tape is the highest spec supported by your drive. Otherwise speed and capacity decrease.

(I use this http://www.fujifilmmediasource.com/specs/new/dlt/dltdrive02.pdf as a reference)

[Wim Van den Wyngaert]

Piet,

Help on backup/dens is not up to date.
Try http://h71000.www7.hp.com/doc/732FINAL/6048/6048pro_001.html#startsubcommand_120 to find the correct setting.
It is SDLT that you should use, I think.

Wim

[Martin P.J. Zinser]

Hello Piet,

poking into the DCL tables with Verb gives the follwoing valid values for density:

define type MDFMT_DENSITIES
keyword DEFAULT
keyword 800
keyword 1600
keyword 6250
keyword 3480
keyword 3490E
keyword 833
keyword TK50
keyword TK70
keyword TK85
keyword TK86
keyword TK87
keyword QIC
keyword TK88
keyword TK89
keyword DLT8000
keyword SDLT
keyword 8200
keyword 8500
keyword 8900
keyword DDS1
keyword DDS2
keyword DDS3
keyword DDS4
keyword AIT1
keyword AIT2
keyword AIT3
keyword AIT4

so it is supposed to work ;-)

Greetings, Martin

[Cass Witkowski]

The backup commands sends the list to a mailbox. Is the process that is reading the mailbox running ok? If it can't read the records fast enough from the mailbox then that could slow down backup one the mailbox fills up.