topic Re: maximum files for VMS BACKUP in Operating System - OpenVMS

maximum files for VMS BACKUP

Jan van den Boogaard — Wed, 27 Jul 2005 08:34:59 GMT

Does anyone know about a maximum number of files on one VMS ODS-5 disk , that VMS BACKUP can still backup with reasonable performance
( 2 MB per sec or more) ?

I ask because we use 36GB disks with many small files. Up to 10 million small files on one disk.

But..... since INDEXF.SYS can be fragmented and large, does VMS BACKUP still perform in this case ?

Re: maximum files for VMS BACKUP

Jan van den Ende — Wed, 27 Jul 2005 08:48:14 GMT

Hallo Jan.

Allereerst: Welkom bij het VMS forum.

As to your question:

Yes, it does perform :-)
But how well?
Prbobably not too fast.

It depends a lot on your hardware config, some on your versions, and a LOT on your SYSGEN settings, and on the UAF settings of the account doing the backup.

So, please specify Disk and Tape Tape hardware and -controllers, the SYSGEN PQL_ parameters, and the settings of your the account doing the backup.

AT least, we then can try to get the best available performance.

But bear in mind, backing up a LOT (10 M ) of small files will inherently last significantly longer than the same amount from just 'a few' files!

Proost.

Have one on me.

jpe

Re: maximum files for VMS BACKUP

Volker Halle — Wed, 27 Jul 2005 09:02:22 GMT

Jan,

you could try for yourself by running a BACKUP/IMA of this disk to the null device, e.g.

$ BACKUP/IMAGE disk: NLA0:NULL.BCK/SAVE

then type a couple of CTRL-Ts and calculate the approximate throughput based on block size, saveset block number and runtime:

Saveset volume:1, saveset block:267 (32256 byte blocks)

You can abort the BACKUP at any time after you've calculated a typical throughput after a couple of minutes. And you could also run it to completion. Irregardless of the output device used lateron for a real backup, it won't get any faster (except maybe by tuning).

Volker.

Re: maximum files for VMS BACKUP

Jan van den Boogaard — Wed, 27 Jul 2005 09:53:33 GMT

Hi Jan.
Thanks for your welkom!

We use Alphaservers ES45, OpenVMS 7.3-2 and disks from EVA-5000 SAN.

The tape units we have not decided yet, MSL5000 or maybe ESL712 directly connected to the Alpha.

These are the UAF quota for the account doing the backup:
Maxjobs: 0 Fillm: 100 Bytlm: 170000 Maxacctjobs: 0 Shrfillm: 0 Pbytlm: 0
Maxdetach: 0 BIOlm: 150 JTquota: 4096 Prclm: 10 DIOlm: 150 WSdef: 150000
Prio: 4 ASTlm: 250 WSquo: 450000 Queprio: 0 TQElm: 120
WSextent: 450000 CPU: (none) Enqlm: 2000 Pgflquo: 3000000

And the PQL SYSGEN params are in the attachment.

Which quota you think has the largest impact on the thruput?

Re: maximum files for VMS BACKUP

Jan van den Boogaard — Wed, 27 Jul 2005 09:53:51 GMT

Re: maximum files for VMS BACKUP

Ian Miller. — Wed, 27 Jul 2005 10:08:53 GMT

here are the current recommendations
http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/00/01/117-con.html#proc-sec

Lots of small files will certainly be slower than fewer large ones.

Re: maximum files for VMS BACKUP

Jan van den Ende — Wed, 27 Jul 2005 10:24:06 GMT

So, Ian again beat me to it.

Use his pointer.

Extra remarks on DIOLM: consider 100 as MAXIMUM.
On many small files, experiment with smaller.
Look out: PQL_MDIOLM if bigger overrules SYSUAF!

On RMS_BUFFER: current advise is NOT 127 (max), but a high multiple of 4 (or better, 16) (112 or 124)

Success!

Proost.

Have one on me.

jpe

Re: maximum files for VMS BACKUP

Jan van den Boogaard — Tue, 09 Aug 2005 05:39:03 GMT

To see how VMS performs, we did a test
backing up 11 million small files on 1 disk ; 33 GB total, so avg file size 3 kB.

We still had a thruput of 10 GB/hour doing a backup to ESL712 tape.
So not very fast, but still good enough!

Re: maximum files for VMS BACKUP

Karl Rohwedder — Tue, 09 Aug 2005 06:29:25 GMT

A little off topic (maximum files triggers my memory):

Please note that using /INPUT_FILES for the BACKUP command together with /VERIFY fails,
if the input file list specifies files from different disks (BACKUP tries to verify using the diskspec of the last file).

Engeneering has a patch for this for up to lists of app. 100 files. If you want to backup more file using /INPUT_FILES, you have to assign a logical name
BACKUP$_LINKS_IN_INPUT_FILES_LIST
with number of files (this works for up to app. 600-700 files).

regards Kalle