Operating System - OpenVMS
1752815 Members
6052 Online
108789 Solutions
New Discussion юеВ

Re: BACKUP over DECnet, file extensions, performance

 
SOLVED
Go to solution
Wim Van den Wyngaert
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

And I continued with simultanious backups made by the posted script.
1 only : 55 sec
2 (5 seconds after each other) 1m38 sec
3 : 2m22sec

Very small gain but no loss (note : directly allocated the size best-try-cont).

Wim
Wim
Jim_McKinney
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

Wim, did your disk have an extent cache enabled?
Robert Gezelter
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

Wim,

A note of caution is in order. Until one understands WHY the different scenarios behaved differently, it is dangerous to make presumptions.

In this case, I would be very cautious. I would also be very interested in seeing these tests with various settings for the different RMS parameters on the remote node. I would also be interested in an analysis of where the bottleneck is.

- Bob Gezelter, http://www.rlgsc.com
Wim Van den Wyngaert
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

Jim,

To my knowledge space is never allocated contiguously if you allocate it in different parts. At the botom of my old thread
the best result was 4 fragments.

Wim
Wim
Jim_McKinney
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

I'm somewhat surprised, but my testing corroborates what Wim observed. Even without extent caching, and only a single writer to a disk, blocks are not allocated contiguously by default when a large file is created.

$ ld create/siz=100000 test
$ ld connect $1$dga1007:[000000]test.dsk lda7: /AlloClass=12/Share
$ init/system lda7: test
$ mount/noassist/nocache lda7: test
%MOUNT-I-MOUNTED, TEST mounted on _$12$LDA7:
$ dir/size sys$login:named-060_a052.a

Directory DISK$QUORUM:[MCKINNEY]

NAMED-060_A052.A;1 44352

Total of 1 file, 44352 blocks.
$ back/log sys$login:named-060_a052.a $12$lda7:[000000]
%BACKUP-S-CREATED, created $12$LDA7:[000000]NAMED-060_A052.A;1
$ back/log sys$login:named-060_a052.a $12$lda7:[000000];2
%BACKUP-S-CREATED, created $12$LDA7:[000000]NAMED-060_A052.A;2
$ pipe dump/head/block=count=0 $12$lda7:[000000]*.a;* | -
_$ sear sys$pipe $12$,count
Dump of file $12$LDA7:[000000]NAMED-060_A052.A;2 on 1-DEC-2008 11:26:37.74
Global buffer count: 0
File entry linkcount: 0
Count: 5504 LBN: 44496
Count: 976 LBN: 50080
Count: 37920 LBN: 51072
Dump of file $12$LDA7:[000000]NAMED-060_A052.A;1 on 1-DEC-2008 11:26:37.75
Global buffer count: 0
File entry linkcount: 0
Count: 960 LBN: 64
Count: 1024 LBN: 1040
Count: 42416 LBN: 2080
Cass Witkowski
Trusted Contributor

Re: BACKUP over DECnet, file extensions, performance

Jim,

It may be interesting to delete the 2nd version of the file and examine the bitmap to see if there are breaks in the available blocks. Perhaps even dump the header of the index.sys to see where his blocks are located.
John Gillings
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

re reports on experiments
"I'm somewhat surprised"

I'm not surprised. The file system has no memory of what it previously allocated nor any prediction of future intentions. Indeed it has no idea what a particular request will be used for, or even which process it's for, let alone where related extents might be. The allocation is made according to the state of free space and caches at the time of the request. There is no attempt to try to keep files contiguous and given the place of the allocator in the software stack, it wouldn't be feasible to do so for the general case.

On a freshly installed disk, by default, the file system stuff (INDEXF.SYS and friends) will be in the middle of the disk. That gives you two large extents on either side. As you make requests, I believe the allocation algorithms will tend to try to keep the extents even, so the allocations may tend to flip flop between them. This may look like "all over the disk". On the other hand, there are all kinds of other factors which influence where the blocks come from.

Consider, the allocator will do exactly the same thing for a particular sequence of requests, regardless of if the sequence is from one process for one file, or multiple processes writing multiple files.

By definition, you can't "optimize" everything for everyone. So the bottom line is, if you want your files to be contiguous (even though these days it's not entirely clear that it will be any significant benefit), you need to say so up front. Where possible use large initial allocations and large extend sizes to TELL the filesystem your intentions, rather than trying to second guess epiphenomena and emergent behaviour of file system primitives.
A crucible of informative mistakes
Cass Witkowski
Trusted Contributor

Re: BACKUP over DECnet, file extensions, performance

OK, I'll bite. How do you tell Backup or DECnet/RMS your preferences?
John Gillings
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

Cass,

"if possible"...

In general you can jack up default extend quantities. By volume, system wide, or per process. Maybe even consider higher cluster sizes on volumes. Keep telling yourself "disk space is cheaper than time".

For BACKUP operations where the target file(s) is not a saveset, you can use /OVERLAY. For example:

$ CREATE/FDL="FILE;ALLOC 1000000;CONTIGUOUS YES" OUTPUT.DAT
$ BACKUP INPUT.DAT OUTPUT.DAT/OVERLAY

With a fairly simple DCL procedure you could create a whole directory tree of preallocated, contiguous files prior to the BACKUP operation.

>but I'm not really thrilled about having
>to add that set rms/extend command in the
>login.com

Why not? What other operations could be performed that can't tolerate a large extent? Many files are truncate on close, so the disk space usage is only temporary.

You could create a special username to perform backups. Given the quota recommendations for high performance backups, you should probably be doing that anyway.
A crucible of informative mistakes
Wim Van den Wyngaert
Honored Contributor

Re: BACKUP over DECnet, file extensions, performance

Redid the test of filling a disk after an init/index=begin. This time mounted with /cache=noext.

The disk fills up like a glass of water. If you delete files the holes stay. When the glass is full, it starts looking for space in the beginning again (finding the holes).
Very neat.

Wim
Wim