show device/window

[Wim Van den Wyngaert]

I have a batch job creating a log file.

Defragmenter tells me that it has 546 extents.

dump/header reports 76 retrieval pointers.

If I do a show dev /window, I see 1 window, total size 144.

As I understand it, I have 76 extents of logfile, all other extents are contiguous with these 76. There should be 11 windows of size 7.

Why is show dev/win showing 1 ?

[Jan van den Ende]

Wim,

Maybe was the disk INITed with a greater /WINDOW (but IIRC the max for that is 80, you have 144)
What is your SYSGEN param ACP_WINDOW ?

You DO get a big performance gain if you manage to get all extents in 1 window, and you reach this gain by setting ACP_WINDOW to 255. The price is memory, can even be tens of kilobytes if there are many fragmented files open simultaniously.
In modern times EVERY VMS system should have this, and you can forever forget about it.

( Engeneering: wouldn't it be a good idea to change the default ? )

Jan

[Willem Grooters]

dump/header reports 76 retrieval pointers>

When the file was created, at least one cluster was allocated and filled. If full, the next cluster would be allocated to that file. Still, there is one fragment - and still one retrieval pointer, but one next more extent.
There comes a point that the next cluetsre to eb allocated is already allocated to another file, so another cluster has to be allocated, somewhere else on the disk. This is the next fragment - and the next retrieval pointer. And so on, and so on.
In the end, the file has been extended 545 times, and results in 76 fragments on disk.
Without more information (indexf.sys!) you cannot tell how these fragments are spread over the disk.

Next is just a wild guess but for what I knwo of VMS and RMS, I think roughly accurate (I hope).
Number of windows is the number of channels open to that file by that particular PID, and sixe the number of blocks kept in memory (cache or in the process's buffer).
That menas: the process will see just 144 blocks of the file at a time. If you had opeend another channel to the same file in the same PID, you would have 2 windows.

Willem

[Willem Grooters]

Jan,

I just did the same thing on "my" disks and found most 80 and 144, some 112 and 128. Indexf.sys has 192 as windowsize.
Some disks have mostly 80, others mostly 144.
A quick scan (not extensively!) revealed:

* executables ==> 80 (at least: most of them)
* datafiles, .COM, .LOG ==> 144
* indexf.sys ==> 192
* sysdump.dmp ==> 128

As far as .EXE are involved: just a VERY few have a windowsize > 80 (112, 128, 144)

BTW: These numbers look the same for ODS-5 and ODS-2 disks. I don't know if clustersize and extend quantity matter.

[Jan van den Ende]

Willem,

you woke me up.

The story is more or less yours and mine multiplied.
Yes, each extent pointer is stored in the header. When the header is full, you get an extension header (etc...)
And what the a WINDOW is holding, is a number of extension headers...
So, Wim, if you have 1 window, that means the number of HEADERS is less then the window-size. So, you have to do quite a lot of fragmenting to get your headers not fitting in the window view, BUT IF YOU DO, THE PENALTY IN PERFORMANCE IS QUITE SEVERE!!
(that's to say, if you move randomly through the file).

And now I hope I have not forgotten still more...

Jan

[Hein van den Heuvel]

> Defragmenter tells me that it has 546 extents.
> dump/header reports 76 retrieval pointers.

I would like to understand that better.
1 fragment = 1 extent in my book.
546 extends would mean several file headers... that sucks.
Are you ignoring extention headers?
Maybe you can show a little bit of a header dump?

I really thought I remembered that adjacent extents would be 'glueed' together by the xqp in the file header, not just in the window control block.

For sequental read/write (like what you do to a batch job log) is really does not matter much whether the window holds all pointers or not, as you never come back and just use 1. Still, every time a pointer is not there, a header needs to be visited (in the header cache, possibly needing an IO).
(btw... There is only one channel/file)


Instead of worrying about how to deal with windows and fragments, you should perhaps focus on trying to avoid them. How about
SET RMS/SYST/EXTE=2000 ?
or
SET VOLUME/EXTE=2000 logfiledisk:

Myself I am a big fan of using a cluster size of 512 or 1024 or so.
Obviously this may be too costly for applications with hundreds of thousand of files on a disk, but if you have less than 10,000 files, it may be worth the waste.
Just do your own calculation: cluster size = acceptable_blocks_wasted divided by largest_number_of_files_expected.
It does not have to be precise.
It does not have to be a multiple of 2.

Hope this helps some,
Hein.

[Wim Van den Wyngaert]

Hein,

Again, just trying to understand things.
I posted the output of a simular file.

Wim

[Hein van den Heuvel]

That looks broken. I would trust DUMP/HEAD and distrust the DFO data. Odd. Very odd.

Hein.

[Willem Grooters]

Wim,

Just to get the whole picture, can you supply output of

$ SHOW DEV/FULL

and

$ DIR/FULL

Willem

[Wim Van den Wyngaert]

Voila

[Hein van den Heuvel]

I notice you have DFO E2.7


I just installed DFO V2.7 [PLUS]. It seems older "2002" but it works for me.
See below.

I don't like the "E" in your version. It may be what you need, but I'd verify.
I pulled "V" from: http://www.support.compaq.com/sms/pfo/

You may also want to compare with DFU output.

Hein

$ defrag show sys$login/vol
Disk File Optimizer for OpenVMS DFG V2.7
© 2002 Compaq Information Technologies Group, L.P
:
Most Fragmented File: [X]NEWFOLDER.PST;1 (1137 extents)

$dump/head /bloc=coun=0 [X]NEWFOLDER.PST;1/out=bad.tmp
$ sea/stat bad.tmp "LBN:" /win=0
U$1:[HEIN]BAD.TMP;1

Records searched: 1973
Records matched: 1137
(16 headers

[Wim Van den Wyngaert]

Hein,

DFU 2.6 also reports 27 extents.

Wim

[Helmut Ammer]

Wim,

in your first attachment Defrag says:
Most Fragmented File:
[OPERATIONS.SYSMGR.WVW]WIM2.LOG;2 (27 extents)
but the header dump is for OPS$MGR:[WVW]WIM2.LOG;3

In your second attachment your file is WIM.LOG;1

You can also use DFU (freeware CD) and do a
SEARCH disk/FILE=file/FRAGMENT
to check the number of extents.

Viele Gruesse
Helmut

[Wim Van den Wyngaert]

Helmut


I posted the output of a simular file.


My original file was already deleted.

Wim

[Helmut Ammer]

Wim,

please check again! In your first attachment Defrag says file with version number 2 (TWO) has 27 extents. Then you dumped the header of version number 3 (THREE) which has 14 extents.
Different files -> different number of extents!

I checked it on my system and got always the same number of extents (using Defrag or counting retrieval pointers of using DFU).

What you see in column "Win Size" of SHOW DEV/WIND is the size of the WCB in bytes! It is not the number of extents. Each file has one WCB. Per default the WCB can map 7 retrieval pointers, which results in 144 bytes. If the file has more than 7 extents a window turn is performed, to map the next 7 pointers. No additional WCB is created.

If one extent is located next to another extent, they are not "contiguous" (virtual blocks) within the file!

For more info please see:
OpenVMS File Systems Internals from Brian Schenkenberger (Elsevier)
ISBN 1555582699

Helmut

[John Eerenberg]

Some practical information.

When you do a show device /window, if you see a "1C" (is it always "1C"?)under "Win Cnt" the file is in cathedral windows mode and if the file is contiguous, the "Win Size" will be 80 bytes. Technically the Window Control Block (WCB) does not use all 80 bytes, only 68. The left over 12 bytes are for the second LBN which doesn't exist if the file is contigous. All of this is to put it into one block allocated from nonpagedpool which is done in multiples of 16 bytes. So what happens if there are 3 extents? The used "Win Size" is 92 but the show device command reports it as 96. If the file has dozens of extents and "Win Cnt" says "nC", (n=1 many times) then all the WCB's are read into memory and held in nonpagedpool (watch for pool expansion).

Since the mount command only goes up to 80 retrieval pointers, coding an application to open files in cathedral mode pulls in all the retrieval pointers. There are utilities to set a volume to be mounted *as if* /windows=255 (which mean cathedreal windows are enabled for the volume) is used on the mount command. I only do this for disks that I know the fragmentation characteristics so I manage nonpagepool properly.

Also, cathedral windows can be important in the follow case: When a contiguous file is 7 * 65536 blocks long or greater (and a mount /window is not specified) there will be window control block turns even though the file is contiguous. This really hits home for files that are contiguous and 80 * 65536 blocks or more. Say I have a file that is 57,671,600 blocks and the disk is mounted /window=80, I am going to do IO performing window turns when accessing the various records that are outside a given 80 * 65536 block area (in this example there are 11 of them). The way to have the window turns reduced (e.g., see $ monitor fcp) is to set the volume or open the file with cathedral windows.

[Wim Van den Wyngaert]

John,

Thank for the complete info. Especialy the large files section. This explains my window turn rate I see now.

Helmut,

I recreated the file and counted "count:" in the dump output. 585 retrieval pointers.
dump/head/block=count=0 wim.log/out=wim.lis
sea wim.lis count:/stat/noou

Files searched: 1 Buffered I/O count: 5
Records searched: 920 Direct I/O count: 2
Characters searched: 41158 Page faults: 30
Records matched: 585 Elapsed CPU time: 0 00:00:00.01
Lines printed: 0 Elapsed time: 0 00:00:00.01



Defragmenter reports 579 extents.
Most Fragmented File:
[OPERATIONS.SYSMGR.WVW]WIM.LOG;2 (579 extents)


DFU reports 77 fragments.
Most fragmented file :
WSYS01$DKA0:[OPERATIONS.SYSMGR.WVW]WIM.LOG;2 ( 19102/19104 blocks; 77 frag
ments)