backup qualifiers

[Marc Van den Broeck]

To copy a large file (4 Gb) from disc to disc, backup seems to be faster then copy. Are there qualifiers to speed up the copy or the backup process?

Thx
Marc

[Jeroen Hartgers_3]

$ backup

example

$ backup/log test.db -
dka100:[000000.dir1.dir2.dir3]*.*

the log qualifier will genarte the message:
%BACKUP-S-CREATED, created dka100:[000000.dir1.dir2.dir3]test.db

[Marc Van den Broeck]

Jeroen,

I guess using the log qualifier will not speed up the process?

Rgds
Marc

[Karl Rohwedder]

You could try different values of
SET RMS /BLOCK=? /BUFFER=?, but one limiting factor will be the fragmentation of the source disk.
For Backup see the various streams about backup performance.

regards Kalle

[Jeroen Hartgers_3]

the log qualifier will speed down but in this case with 1 file nor realy. if there is a lot of log output by examble een backup restore of a disk it will speeddown the back restore for a match time.

[Jeroen Hartgers_3]

that's right

[Jan van den Ende]

Marc,

You are DEFINITELY right! /LOG wil slow it down. For each file processed, an output line has to be constructed, and an extra IO must be executed to write that line to SYS$OUTPUT or the specified logfile.
But in this case, an operation on just one file, the effect is very minimal.

@Jeroen:

...
dka100:[000000.dir1.dir2.dir3]*.*


You are playing with matches in a gunpowder storage!
Specifying [000000... in a backup output string generates a toplevel 000000.DIR.
So far, no noticable problem yet, maybe for a long time
BUT, if you process this directory the same way again, you will have created 000000.000000.DIR
And NOW you are in for some fun!

Please, train yourself to NOT use the [000000.xyz syntax in output specifications!

fwiw,

Proost.

Have one on me.

jpe

Proost.

[Ian Miller.]

In VMS V8.2 and V7.3-2 patch kits the default block size for copy was increased to 127 blocks and this has helped copy performance.
The optimim value if using EVA is 124.
If you are not at these versions then use the SET RMS command to increase the default multiblock and multibuffer counts.
Try MBC=124 and buffer count=3 for a start.

[Marc Van den Broeck]

Jan,

my /log remark was ment a little bit sarcastic because the answer of Jeroen was not really answering my question (Jeroen, dont feel offended).
So, dont think nor talk about /log but about qualifiers speeding up the process ;).

Rgds Marc

[Galen Tackett]

/LOG will REALLY slow it down if you're using an old ASR33 or similar.
:-)

[Ian Miller.]

Galen, DMC32 and now ASR33 - you are showing your age :-)

[Marc Van den Broeck]

Karl and Ian,

set rms/block=124/buf=3 did not make any (measurable) difference.

(btw: vms version: V7.2-1)

Rgds
Marc

[Jan van den Ende]

Marc,

in effect the speed transfers of large amounts of data are nearly completely dependant on IO throughput.
So, your hardware config is the main factor.
Then, there are some settings that can (de-)optimise the use of the hardware capability.
That optimisation is mainly done with bufferings, and for that, see previous answers.

About being offended: do you really think that someone so easily offendable can do our kind of work for longer than a few months?

For those that sensitive, we have a saying: "If you can not stand the heat, you have to stay out of the kitchen"

fwiw,

Proost.

Have one on me.

jpe

[Karl Rohwedder]

Btw, what performance do you achieve (on what hardware)?

regards Kalle

[Marc Van den Broeck]

Karl,

with backup it takes 59' to copy 16Gb of files (15 files between 0.1Gb and 2Gb)
from hsz50 disks to an internal 10000 rpm disc on a COMPAQ AlphaServer DS20E 833 MHz.

(btw: backing it up to a dlt8000 is almost 2 times faster)

Rgds
Marc

[Willem Grooters]

Is your target disk shadowed? If so, it _may_ cause slow down.
/BLOCK= may speed up but could prove contra-productive. I have no measurements - just the feeling.
My memory may have faded and blurred, but as far as I remember, COPY will process a file record by record, and BACKUP will copy block(s)-by-block(s). Also, COPY will use the cache, and BACKUP may bypass it. This would explain yhe difference.

[Jan van den Ende]

BACKUP may bypass cache --
make that BACKUP DOES bypass cache.

I just will have to scan trough my Bootcamp notes to find a direct reference, but it has been pointed out in an XFC discussion.

Proost.

have one on me.

jpe

[Volker Halle]

Marc,

with your backup disk-to-disk, you reach about 4.5 MB/sec. If it's twice that fast disk-to-tape, then the output disk probably seems to be the bottleneck.

You could try to disable high-water-marking on the destination disk (SET VOL/NOHIGH), it reduced the duration of my little test by about 10%.

Volker.

[Jan van den Ende]

Marc,

just to make sure, regarding Volkers answer:

ARE the input and output disk 2 separate physical devices?
If so, are they on the same or different controllers? Of what type?

Proost.

Have one on me.

jpe

[Marc Van den Broeck]

Volker,

disabling high-water-marking on a disc is one of the first things I do on a disc I have to use.

Jan,

from hsz50 disks to an internal 10000 rpm disc on a COMPAQ AlphaServer DS20E 833 MHz.


Rgds
Marc

[Hein van den Heuvel]

For single large file copy, the preferred tool ought to be copy, and it has proven to be faster than backup in a few tests.

What VMS version? Copy increased its IO size in recent version and with that ought to be the faster tools.

The whole exercise is rather dependend on all sorts of parameters I am afraid. Backup will issue a batch of IOs one the input side to fill its IO buffers and then wait for them all all to complete, next start to write synchroneously. No overlap :-(. (Yes, there is room for improvement...)
The batch size depends on process quotas like ASTLM, DIRIOLM, WS,...

Copy simply does read, write, read, write....

It will depend on things like fragementation and disk controller smarts which one will win. The controller may recognizes the pattern and starts read-ahead for the copy case, but the multiple IOs for backup may muddy the waters. Also, the deep queue that backup can create can throw controllers into a temporary state of shock it seems :-).

Hope this helps some,
Hein.

[Marc Van den Broeck]

Hein,

thx for info.
vms version: V7.2-1

Rgds
Marc

[Hein van den Heuvel]

We had an internal report/experience where BACKUP from an XP was slow compared to copy.

There the deep queue / high parallelism saturated the XP front end / cache algoritmes.

The workaround was to drop DIOLM and PQL_MDIOLM to 8 from the current recommendation of 100. The IOPS jumped from 300 to 2,000.

THey also found that the XP, and I imageing other cached controllers also, really like IO sizes in powers of 2, and transfers alling at powers of 2. On VMS that would suggest clustersizes of 4, 8, 16, 32, 64 to at least get the first IOs started alligned.

fwiw,
Hein.

[comarow]

Large cluster sizes on the backup device will help. Huge cluster size.
/block=32768 would be a good starting size.

The original files should be defragmented.

Now, depending on the device to be backed up to, working sets, diolim are changed.
For example keep diolm to <32 on a san disk or the cache will thrash, and 4096 on a scsi disk.


We need the devices and controllers and files.

[Wim Van den Wyngaert]

/block won't help because it is a save set qualifier.

Avoid raid-5 on the destination disk and monitor process quota.

I vaguely remember that very high quota may decrease performance.

Wim