BACKUP over DECnet, file extensions, performance

[Cass Witkowski]

Jim,

It may be interesting to delete the 2nd version of the file and examine the bitmap to see if there are breaks in the available blocks. Perhaps even dump the header of the index.sys to see where his blocks are located.

[John Gillings]

re reports on experiments
"I'm somewhat surprised"

I'm not surprised. The file system has no memory of what it previously allocated nor any prediction of future intentions. Indeed it has no idea what a particular request will be used for, or even which process it's for, let alone where related extents might be. The allocation is made according to the state of free space and caches at the time of the request. There is no attempt to try to keep files contiguous and given the place of the allocator in the software stack, it wouldn't be feasible to do so for the general case.

On a freshly installed disk, by default, the file system stuff (INDEXF.SYS and friends) will be in the middle of the disk. That gives you two large extents on either side. As you make requests, I believe the allocation algorithms will tend to try to keep the extents even, so the allocations may tend to flip flop between them. This may look like "all over the disk". On the other hand, there are all kinds of other factors which influence where the blocks come from.

Consider, the allocator will do exactly the same thing for a particular sequence of requests, regardless of if the sequence is from one process for one file, or multiple processes writing multiple files.

By definition, you can't "optimize" everything for everyone. So the bottom line is, if you want your files to be contiguous (even though these days it's not entirely clear that it will be any significant benefit), you need to say so up front. Where possible use large initial allocations and large extend sizes to TELL the filesystem your intentions, rather than trying to second guess epiphenomena and emergent behaviour of file system primitives.

[Cass Witkowski]

OK, I'll bite. How do you tell Backup or DECnet/RMS your preferences?

[John Gillings]

Cass,

"if possible"...

In general you can jack up default extend quantities. By volume, system wide, or per process. Maybe even consider higher cluster sizes on volumes. Keep telling yourself "disk space is cheaper than time".

For BACKUP operations where the target file(s) is not a saveset, you can use /OVERLAY. For example:

$ CREATE/FDL="FILE;ALLOC 1000000;CONTIGUOUS YES" OUTPUT.DAT
$ BACKUP INPUT.DAT OUTPUT.DAT/OVERLAY

With a fairly simple DCL procedure you could create a whole directory tree of preallocated, contiguous files prior to the BACKUP operation.

>but I'm not really thrilled about having
>to add that set rms/extend command in the
>login.com

Why not? What other operations could be performed that can't tolerate a large extent? Many files are truncate on close, so the disk space usage is only temporary.

You could create a special username to perform backups. Given the quota recommendations for high performance backups, you should probably be doing that anyway.

[Wim Van den Wyngaert]

Redid the test of filling a disk after an init/index=begin. This time mounted with /cache=noext.

The disk fills up like a glass of water. If you delete files the holes stay. When the glass is full, it starts looking for space in the beginning again (finding the holes).
Very neat.

Wim