topic Re: File allocation question in Operating System - OpenVMS

File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 04:25:59 GMT

I defragmented the system disk of a 7.3 system with backup/image.

After the first boot I find that dfg show dsa0/vol shows 1.2 Mblocks as the maximum free extent while the free space is 1.5 Mblocks. I checked the volume map in defragmenter and found some files in the "free zone" but not allocated in the beginning but somewhere after 20% of the free space. So, the contig space is lost very fast.

Why is VMS not allocating these files in the beginning ? What is the allocation strategy ?

Wim

Re: File allocation question

John Travell — Tue, 03 Jul 2007 06:45:20 GMT

On a default disk init the index file is put in the middle of the disk. This means that whatever else is put around it, you have a minimum of two free spaces unless you manage to exactly fill one of them.
Could those files you see 'after 20% of the free space' be the index file and others deliberately placed in the middle of the disk ?
JT:

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 06:46:34 GMT

No. I did a init/index=begin. And then backup/noinit.

Wim

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 06:59:58 GMT

The newly created files during the boot were allocated in the "first 20%" zone. Defrag reports their names if you click on the zone. I now found that after some uptime, files were also created in the "last 20% zone of the free space. And VMS seems to be filling these 2 zones.

Wim

Re: File allocation question

Jon Pinkley — Tue, 03 Jul 2007 07:25:35 GMT

Wim>>>The newly created files during the boot were allocated in the "first 20%" zone. Defrag reports their names if you click on the zone. I now found that after some uptime, files were also created in the "last 20% zone of the free space. And VMS seems to be filling these 2 zones.
+++++++++++++++++++++++++++++++++++++++++++

I think this is related to the extent cache.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1136790

It would be easy to test a non-system disk by mounting/nocache, but I don't know if there is a way to disable cache on a system disk. Even if you could, I don't think you want to run normally with nocache in effect.

Are the two areas being filled in by different nodes in the cluster? i.e. the first 20% by the first node, and last 20% zone by the second node? If I remember correctly, each node has its own extent cache locked (so they will not be allocated by another node without that node expressly asking for the space).

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 07:28:29 GMT

Jon : not a cluster. Just an AS500 with 2 disks forming dsa0.

Wim

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 07:56:28 GMT

Good stuff to read(esp. ch 2). I think the reason must be extent cache related but I'm not sure yet.

acp_extlimit is on 100% on my node.

http://www.diskeeper.com/fragbook/contents.htm

Wim

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 08:27:40 GMT

Correction. Missed the /10. It's 10%.

Wim

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 08:31:50 GMT

Jon : this is what you mean (out of doc of 1992)

Also, in cases of large VAXClusters where many nodes are sharing
a disk, the maximum blocks in extent cache may be less than the
limit set by ACP_EXTLIMIT. An additional factor is used in these
cases in order to prevent over commitment of the extent cache.
Briefly, the free space on the disk is divided by the number of
nodes mounting the disk (actually, the number of locks taken out
on the disk +2). If the result of this equation is less than the
result of the ACP_EXTLIMIT equation, then the smaller value is
used as the maximum size of extent cache.

Re: File allocation question

Hein van den Heuvel — Tue, 03 Jul 2007 08:35:05 GMT

The ext_limit is in pro-mille, so 100 is effectively 10% of the freespace. That's not equal to, but not too far either from the 20% you mention. It is not unlikely to match the mount-time pre-load of the extent cache... but I can not readily find that documented.

The other param in this area is ofcourse ACP_EXTCACHE which indicates how many entries. With large contig free space areas, there will be few extents to exceed the extlimit.

The extent cache basically tries to avoid scans for space in bitmap.sys. If this part of the application is concerned with just allocating a few large, as contiguous as possible files, then the ext cache is of no help to it and the disk(s) involved should be mounted /CACHE=(NOEXTEN,LIMIT=0)

If the application is concerned with is single large allocation, then you may want to trigger a flush of the extent cache just before. Privved code could jiggle the appropriate F11B$C lock directly. Normal folks can just use $COPY/CONT/ALLO=9999999 NL: device: to tell the CACHE_SERVER process to go do its thing.

For educational purposes one could use ANAL/SYSTEM anc look in the Volume Control Block for VCB$L_CACHE to find the link of cached extent.

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 08:40:56 GMT

Hein,

This still doesn't explains why my contig space decreased from 1.5 Mbl to 0.9 Mbl without having anything serious active (some idle processes and some processes generating low volume log).

Wim

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 09:59:00 GMT

Experiment on a single node.

Create file of 1000 blocks every 0.1 second. With copy without /contig and monitor volume map. Disk was defragmented and about 10% used.

It starts allocating the space evenly spread over the disk. About 20 spots are created.

Then it starts extending about 10 of these spots. Until they take up about 1% of the disk each. Then it starts extending the other sports.

My conclusion : VMS tries to spread the files over the complete free space. Not trying to keep contig space.

Tried it in a cluster too. Same reaction.

Wim

Re: File allocation question

Wim Van den Wyngaert — Tue, 03 Jul 2007 10:33:38 GMT

Remounted the disk with 100% freespace in cache instead of 10%.

Space is now allocated in 4 spots instead of 20. And the spots are all extended cleanly. Would thus lead to better preservation of contig space.

Still curious for the theory behind it.

Wim