1828370 Members
3064 Online
109976 Solutions
New Discussion

Re: backup qualifiers

 
Marc Van den Broeck
Trusted Contributor

backup qualifiers

To copy a large file (4 Gb) from disc to disc, backup seems to be faster then copy. Are there qualifiers to speed up the copy or the backup process?

Thx
Marc
31 REPLIES 31
Jeroen Hartgers_3
Frequent Advisor

Re: backup qualifiers

$ backup

example

$ backup/log test.db -
dka100:[000000.dir1.dir2.dir3]*.*

the log qualifier will genarte the message:
%BACKUP-S-CREATED, created dka100:[000000.dir1.dir2.dir3]test.db
Marc Van den Broeck
Trusted Contributor

Re: backup qualifiers

Jeroen,

I guess using the log qualifier will not speed up the process?

Rgds
Marc
Karl Rohwedder
Honored Contributor

Re: backup qualifiers

You could try different values of
SET RMS /BLOCK=? /BUFFER=?, but one limiting factor will be the fragmentation of the source disk.
For Backup see the various streams about backup performance.

regards Kalle
Jeroen Hartgers_3
Frequent Advisor

Re: backup qualifiers

the log qualifier will speed down but in this case with 1 file nor realy. if there is a lot of log output by examble een backup restore of a disk it will speeddown the back restore for a match time.
Jeroen Hartgers_3
Frequent Advisor

Re: backup qualifiers

that's right
Jan van den Ende
Honored Contributor

Re: backup qualifiers

Marc,

You are DEFINITELY right! /LOG wil slow it down. For each file processed, an output line has to be constructed, and an extra IO must be executed to write that line to SYS$OUTPUT or the specified logfile.
But in this case, an operation on just one file, the effect is very minimal.

@Jeroen:

...
dka100:[000000.dir1.dir2.dir3]*.*


You are playing with matches in a gunpowder storage!
Specifying [000000... in a backup output string generates a toplevel 000000.DIR.
So far, no noticable problem yet, maybe for a long time
BUT, if you process this directory the same way again, you will have created 000000.000000.DIR
And NOW you are in for some fun!

Please, train yourself to NOT use the [000000.xyz syntax in output specifications!

fwiw,

Proost.

Have one on me.

jpe

Proost.

Don't rust yours pelled jacker to fine doll missed aches.
Ian Miller.
Honored Contributor

Re: backup qualifiers

In VMS V8.2 and V7.3-2 patch kits the default block size for copy was increased to 127 blocks and this has helped copy performance.
The optimim value if using EVA is 124.
If you are not at these versions then use the SET RMS command to increase the default multiblock and multibuffer counts.
Try MBC=124 and buffer count=3 for a start.
____________________
Purely Personal Opinion
Marc Van den Broeck
Trusted Contributor

Re: backup qualifiers

Jan,

my /log remark was ment a little bit sarcastic because the answer of Jeroen was not really answering my question (Jeroen, dont feel offended).
So, dont think nor talk about /log but about qualifiers speeding up the process ;).

Rgds Marc
Galen Tackett
Valued Contributor

Re: backup qualifiers

/LOG will REALLY slow it down if you're using an old ASR33 or similar.
:-)
Ian Miller.
Honored Contributor

Re: backup qualifiers

Galen, DMC32 and now ASR33 - you are showing your age :-)
____________________
Purely Personal Opinion
Marc Van den Broeck
Trusted Contributor

Re: backup qualifiers

Karl and Ian,

set rms/block=124/buf=3 did not make any (measurable) difference.

(btw: vms version: V7.2-1)

Rgds
Marc
Jan van den Ende
Honored Contributor

Re: backup qualifiers

Marc,

in effect the speed transfers of large amounts of data are nearly completely dependant on IO throughput.
So, your hardware config is the main factor.
Then, there are some settings that can (de-)optimise the use of the hardware capability.
That optimisation is mainly done with bufferings, and for that, see previous answers.

About being offended: do you really think that someone so easily offendable can do our kind of work for longer than a few months?

For those that sensitive, we have a saying: "If you can not stand the heat, you have to stay out of the kitchen"

fwiw,

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
Karl Rohwedder
Honored Contributor

Re: backup qualifiers

Btw, what performance do you achieve (on what hardware)?

regards Kalle
Marc Van den Broeck
Trusted Contributor

Re: backup qualifiers

Karl,

with backup it takes 59' to copy 16Gb of files (15 files between 0.1Gb and 2Gb)
from hsz50 disks to an internal 10000 rpm disc on a COMPAQ AlphaServer DS20E 833 MHz.

(btw: backing it up to a dlt8000 is almost 2 times faster)

Rgds
Marc
Willem Grooters
Honored Contributor

Re: backup qualifiers

Is your target disk shadowed? If so, it _may_ cause slow down.
/BLOCK= may speed up but could prove contra-productive. I have no measurements - just the feeling.
My memory may have faded and blurred, but as far as I remember, COPY will process a file record by record, and BACKUP will copy block(s)-by-block(s). Also, COPY will use the cache, and BACKUP may bypass it. This would explain yhe difference.
Willem Grooters
OpenVMS Developer & System Manager
Jan van den Ende
Honored Contributor

Re: backup qualifiers

BACKUP may bypass cache --
make that BACKUP DOES bypass cache.

I just will have to scan trough my Bootcamp notes to find a direct reference, but it has been pointed out in an XFC discussion.

Proost.

have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
Volker Halle
Honored Contributor

Re: backup qualifiers

Marc,

with your backup disk-to-disk, you reach about 4.5 MB/sec. If it's twice that fast disk-to-tape, then the output disk probably seems to be the bottleneck.

You could try to disable high-water-marking on the destination disk (SET VOL/NOHIGH), it reduced the duration of my little test by about 10%.

Volker.
Jan van den Ende
Honored Contributor

Re: backup qualifiers

Marc,

just to make sure, regarding Volkers answer:

ARE the input and output disk 2 separate physical devices?
If so, are they on the same or different controllers? Of what type?

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
Marc Van den Broeck
Trusted Contributor

Re: backup qualifiers

Volker,

disabling high-water-marking on a disc is one of the first things I do on a disc I have to use.

Jan,

from hsz50 disks to an internal 10000 rpm disc on a COMPAQ AlphaServer DS20E 833 MHz.


Rgds
Marc
Hein van den Heuvel
Honored Contributor

Re: backup qualifiers


For single large file copy, the preferred tool ought to be copy, and it has proven to be faster than backup in a few tests.

What VMS version? Copy increased its IO size in recent version and with that ought to be the faster tools.

The whole exercise is rather dependend on all sorts of parameters I am afraid. Backup will issue a batch of IOs one the input side to fill its IO buffers and then wait for them all all to complete, next start to write synchroneously. No overlap :-(. (Yes, there is room for improvement...)
The batch size depends on process quotas like ASTLM, DIRIOLM, WS,...

Copy simply does read, write, read, write....

It will depend on things like fragementation and disk controller smarts which one will win. The controller may recognizes the pattern and starts read-ahead for the copy case, but the multiple IOs for backup may muddy the waters. Also, the deep queue that backup can create can throw controllers into a temporary state of shock it seems :-).

Hope this helps some,
Hein.
Marc Van den Broeck
Trusted Contributor

Re: backup qualifiers

Hein,

thx for info.
vms version: V7.2-1

Rgds
Marc
Hein van den Heuvel
Honored Contributor

Re: backup qualifiers


We had an internal report/experience where BACKUP from an XP was slow compared to copy.

There the deep queue / high parallelism saturated the XP front end / cache algoritmes.

The workaround was to drop DIOLM and PQL_MDIOLM to 8 from the current recommendation of 100. The IOPS jumped from 300 to 2,000.

THey also found that the XP, and I imageing other cached controllers also, really like IO sizes in powers of 2, and transfers alling at powers of 2. On VMS that would suggest clustersizes of 4, 8, 16, 32, 64 to at least get the first IOs started alligned.

fwiw,
Hein.
comarow
Trusted Contributor

Re: backup qualifiers

Large cluster sizes on the backup device will help. Huge cluster size.
/block=32768 would be a good starting size.

The original files should be defragmented.

Now, depending on the device to be backed up to, working sets, diolim are changed.
For example keep diolm to <32 on a san disk or the cache will thrash, and 4096 on a scsi disk.


We need the devices and controllers and files.
Wim Van den Wyngaert
Honored Contributor

Re: backup qualifiers

/block won't help because it is a save set qualifier.

Avoid raid-5 on the destination disk and monitor process quota.

I vaguely remember that very high quota may decrease performance.

Wim
Wim