INITIALIZE /INDEX=xxxxx

[Art Wiens]

I didn't want to hijack Peter's thread about "disk to disk to tape", but I was just curious about Hein's intent on wanting the index file at the beginning of the disk.

Assuming your not dealing with physical drive = volume ie. raidsets, logical drives carved out of an array, LD container files etc., does it really matter anymore where the index file is?

I believe having the index file at the beginning or middle or end was more an attempt to optimize/reduce disk head seek time, was it not? If there are many heads on many disks involved. is it not "moot"?

We are about to build new ES80's using storage presented from an EVA8000 ... just curious if I should take the /index switch into consideration when init'ing the new "disks".

Cheers,
Art

[Steven Schweda · Accepted solution]

> [...] does it really matter anymore where
> the index file is?

It does if you're planning "to 'truncate' the
container file".

[Hoff]

Initializing a disk is covered in http://64.223.189.234/node/193

I'd be looking at the cluster factor (minimum of 16, or a multiple of 16) and at allocating sufficient headers, and at enabling dynamic volume expansion.

The index file will typically want to be in the middle of the disk; at its default position. Yes, due to the perceived advantages in disk seeks for such a frequently-accessed file. Yes, caching can potentially eliminate the positioning benefits -- further, with multiple heads,

Stephen Hoffman
HoffmanLabs LLC

[Hein van den Heuvel]

Steven>> It does if you're planning "to 'truncate' the container file".

:-)

Give the man 10 points!


>> I didn't want to hijack Peter's thread about "disk to disk to tape",

Excellent.
Future readers... that would be:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1136467

>> does it really matter anymore where the index file is?

Is does if you want maximum contiguous space!

>> believe having the index file at the beginning or middle or end was more an attempt to optimize/reduce disk head seek time, was it not?

Right. The assumption was that the dominant default application / usage case would be fill up disks with lots of smallish files (application development environment). Thus going from the file header to its data would on average involve a head seek over half of half the full seek distance.

The price for that is to force half disk seeks for near empty disks as they are being filled. And near half disk seeks at the disk fills up to get to the more recent files which are likely to me more popular.

The assumption was that non average applications, for example deploying file occupying 1/4 or more of the disk, would be smart enough to force their own INIT/INDEX.
And for a few years (decades) they were!

Nowadays I suspect that is a lost art, and I feel, like Art, that is matters much less, if any.

The specific case of the EVA?... well when you create a virtual unit (disk) the blocks are reserverd, but not actually allocated. The allocation happens in mega-byte sized chunks (4mb storage units?) on first touch, which would be the INIT. So when starting on completely clean eva disk group, even if /INDEX=MIDDLE is in effect, the actual LBNs on the disks will be the low blocks!.

(For repeatable benchmarks one has to write a pattern from LBN 0 to the end to force the actual allocation on EVA's!)

I tend to ignore /INDEX=xxx and if i do think about it, then i tend to select /index=begin

fwiw,
Hein.

[Robert Gezelter]

Art,

I would take a moderate position on this entire question.

WADR to Hein, I would not often locate the INDEX file at the beginning of the disk. Given current disk capacities, I would also not tend to choose the absolute middle, for the reasons already mentioned.

The discussion about the history has already been mentioned, and I believe that it was cited correctly.

What DOES matter, is the (expected) file usage. If I am initializing a disk that is to be carved up solely into a pile of LD container files, putting the Index file at the beginning is not a real problem. If I know that the disk will already be half full when I start, the middle might be a better choice.

On an otherwise empty disk, that will gradually fill? Probably I would place it some percentage of the way in, with the expectation that I would be expanding the volume at a later date (see my presentation on expanding volumes without interrupting users at http://www.rlgsc.com//hptechnologyforum/2005/1146.html ).

Absent some hard statistics, this is an almost unresolvable question. The presence of caches at various places (device, controller, host, XQP, RMS) tends to obscure, rather than clarify the issue. Personally, I try to analyze issues without the assumption of caches: decisions that are good tend to get better with caching.

My US$ 0.02.

- Bob Gezelter, http://www.rlgsc.com

[John Gillings]

Art,
My 2c too...

Yes, the original intent for placement of the index file was seek times, and yes, it's not that relevant in the context of some current technologies - LD, partitioning, EVA, caches, etc. By default, allocations would be taken from the largest contiguous chunk, so filling a disk would tend to grow outwards, alternating sides for equal sized allocations. This was good for seeks because it tended to keep the disk well localised (though a large file with small extensions could get distributed further and further apart).

As Hein pointed out, one reason to choose "BEGINNING" or "END" would be to maximise the (possibly virtual) contiguous space on the volume.

I'd say something that's potentially more important than the location of INDEXF.SYS is to specify a reasonable estimate for its initial size using the /HEADERS qualifier. Regardless of storage technology, extent slots in the header of INDEXF.SYS are a VERY finite resource. Granted, the post V6.2 algorithm for extending INDEXF.SYS is far better than the previous one, but an underestimate on the INIT (or default /HEADERS=16) risks pushing a volume into HEADERFULL state.

Unless you have good reasons for choosing otherwise, I'd tend to go with the default "MIDDLE", if for no other reason than to avoid potential pathological or boundary states that may not have been well exercised.

(FYI, I recall a weird case where copying the entire contents of one particular release of an OpenVMS documentation CD using a specific model of CD drive resulted in a long seek which confused the seek logic in the drive. The COPY failed, but would be OK if individual files were copied. Although this had nothing to do with placement of INDEXF.SYS, it shows how particular access patterns can result in unexpected errors)

[Hoff]

The logical middle won't be at the half-stroke position. Depending on the numbers of platters and on numbers of sectors present at each track, the logical middle could easily be at the extreme end of a head stroke.

With four platters of equal capacity and consistent linear block allocation per track, the logical middle will be at the physical edge of the disk; in the "worst" position.

[David Jones_21]

Back when my site was using VDDRIVER to split large RAID volumes into virtual drives, placing the index file at the beginning made packing the large contiguous files into the LBN range much simpler.

Disks are so large now, there's little relative space penalty for simply initializing the volume with 16 million headers.

[Dean McGorrill]

we used to init a little before the middle so the bulk of the index would reside over the real middle. now I just
preallocate the headers & directories, happy
the days of rm05's are over.

[Jon Pinkley]

Art,

My experience is that the output disk of a BACKUP/IMAGE restore uses the space after the index before using the space before the index. It doesn't try to allocate "nearest" to the middle. My guess is that it has a pointer to the next free location, and that pointer is initialized to the first free space after the end of the Indexf.sys file.

This can be verified by using DFU report/graph immediately after restoring a disk. It is most obvious if the disk is 50% full or less.

The point is that when doing an image restore, the lowest available blocks are not what is chosen. I can't say whether the initial free space after the indexf.sys is initially larger or smaller.

One other note: File positioning information is not preserved by an image backup, even when going to a disk of the same size (this is my observation from BACKUP 7.2-2 system, I haven't tried on 7.3-2 or 8.3). Also, if you have large files that must (or you want to be) contiguous, put them in a directory that will be restored early. For example, if you have a contiguous file that is 40 percent of the volume, you will probably want it in a directory like [000BEG] so it will be processed early. Or avoid the problem by using /INDEX=BEGINNING.

Regarding your question about EVA8000 and positioning the /INDEX: Since the allocation of psegs to drives is under the control of the EVA, even if you preallocate as suggested by Hein, I am not certain that any order you have will be preserved after "leveling" occurs on the EVA.

Having the index in the default middle position shouldn't affect performance one way of the other.

Supposedly, the EVA has or will have the ability to "shrink" volumes, because Windows servers have the ability. That may be a reason to initialize/index=begin and preallocate headers, just in case you ever want to be able to shrink a volume (if it ever is supported by VMS, or using Hein's proposed method.) This assumes you can use a defragmenter or something else using movefile to free up all allocations in the region to be truncated.

I am planning to user/index=begin when I move from HSZ to EVA, and to change the clustersize to a power of 2.