File header cache size

[Wim Van den Wyngaert]

I'm doing some experiments.

Created 500 directories with 100 files each.

I put ACP_HDRCACHE on 55000.

Did a dir/siz=all of all files. Header hit rate 0% (fine).

Did a 2nd dir (same one). Also 0%.

I don't know the size of a file header but if it's 1 block or less than 2, why is then the hit rate 0 ?

Wim

[Volker Halle]

Wim,

does the 'Maximum buffers in FCP cache' value returned by 'SHOW DEV/FULL disk' reflect your new increased value of ACP_HDRCACHE ?

Does SDA> CLUE XQP/CACHE show an approriately big value in the Poolcnt line in buffer pool #2 (file header blocks are allocated from buffer pool #2) ?

If no, you may need to reboot (maybe after running AUTOGEN ?!). The XQP buffers are allocated from paged pool, which is allocated when the system disk is mounted.

You can look at the cached file header buffers with SDA> clue xqp/Buff=2

To look at the file header buffer itself, use SDA> clue xqp/bfrd= (index from column 1 of previous display). Then use the buffer address displayed and do SDA> EXA bufaddr;200 - this will shown you the file header in memory.

This is all 'interesting stuff' ;-)

Volker.

[Wim Van den Wyngaert]

Volker,

Maximum buffers in FCP cache is 24. So far from 55.000.

Poolcnt is only C.

Clue xqp returns 11 rows only.

It's an AS500 but I didn't use autogen to modify the parameter. If I do a sysgen show it's on 55.000.

???

WIm

[Volker Halle]

Wim,

enter MIN_ACP_HDRCACHE=55000 in MODPARAMS.DAT and run @AUTOGEN GETDATA GENPARAMS to see the effects on PAGEDYN. Then decide, if you want to activeate this parameter using @AUTOGEN GETDATA SETPARAMS and reboot.

Volker.

[Wim Van den Wyngaert]

Did that. Boot is just finished.

Pagedyn is now a lot higher (3.2M was 1.8).

All show/sda commands still hgave the same result.

No message was given during boot.

Wim

[Volker Halle]

Wim,

55000 file headers (ACP_HDRCACHE) would need about 28 MByte of paged pool. One file header = one disk block = 512 bytes.

?!

Volker.

[Ian Miller.]

If you compare old and new versions of SYS$SYSTEMS:PARAMS.DAT what changed?

[Wim Van den Wyngaert]

Small error. Pagedyn is on 32M not 3.2.

Diff :
SYSMWCNT 38121 was 4026
PAGEDYN 32M was 1.8
and some autogen antenna tuning.

Wim

[Volker Halle]

Wim,

now what about:

$ SHO DEV/FULL disk and Maximum buffers in FCP cache ?

SDA> CLUE MEM/STAT ! File System Cache params on 3rd page

$ SHO MEM/POOL should show Paged Dynamic Memory is use 28M or more.

Volker.

[Wim Van den Wyngaert]

Volker,

As said, all show commands result in the same value as before the autogen.

The new commands I will check tomorrow at 8 (German time=Belgian time).

Wim

[Volker Halle]

Wim,

I couldn't get my 96 MB PersonalAlpha Emulator to boot successfully with ACP_HDRCACHE set to 55000, so I couldn't test ;-(

Maybe you start with a slightly lesser value for ACP_HDRCACHE and work your way upwards.

Volker.

[Wim Van den Wyngaert]

Volker,

clue mem/stat shows ****.

Wim

[Volker Halle]

Wim,

as I said, interesting stuff ;-)

SDA> SHOW PARA ACP_HDRCACHE

... ACP$GW_HDRCACHE ...

So it's a WORD ! Maybe start testing with 32000. - the ACP_HDRCACHE field width in SDA> CLUE MEM/STAT is only 4 chars anyway.

SDA> CLUE XQP/CACHE shows the overall cache buffer count in the BFRcnt - same as SHOW DEV/FULL shows in the 'Maximum buffers in FCP cache'

Volker.

[Volker Halle]

Wim,

if BFRcnt is 24. (0x18), the initial allocation of the full block cache in PAGED POOL failed and you just get the MINIMUM number of buffers (reduced block cache).

There should have been a REDCACHE warning message (HELP/MESS REDCACHE) when mounting the system disk during startup.

Volker.

[Wim Van den Wyngaert]

Went for 30.000. Boot failed "insufficient dynamic memory".

Went for 9999. Boot worked.
Maximum buffers in FCP : 10500.
Poolcnt : 270F
.../buf=2 3 lines only (after heavy use).

Did the dir for 11.000 files.
Almost no hits at all in clue mem/stat.

Tried a script that opened the files instead of dir (2 runs). Same result.

I wonder if only open files are kept in it ...

Wim

[Volker Halle]

Wim,

when reading file headers, the XQP reads them into the cache buffers. You are not mounting your disk(s) /NOCACHE ?

Volker.

[Wim Van den Wyngaert]

I rebooted with 30000 again. No message until boot almost complete (decw startup, which failed).
Buf in fcp is at 30500 and no, VMS didn't mount this system disk with nocache.
I get some hits but I think it's for the directory files.

Wim

[Volker Halle]

Wim,

you can check with SDA> clue xqp/buf=2 and you'll see the file headers in the cache.

It will show you (for each file header), the UCB (SDA> SHOW DEV/ADDR=ucb), the LBN and the Lockbasis (=File ID). Then you can find out, which file this header belongs to. You should see lots of file headers in the cache (up to ACP_HDRCACHE).

If I do a DIR/SIZ=ALL disk:[*...], I see those file headers in the cache, but I also do get a File HDR hit rate of 10% or less.

Volker.

[Wim Van den Wyngaert]

Went down to 5000. And the buf=2 shows about 5000 lines. But hit rate on 2nd dir is still a few %.

Wim

[Volker Halle]

Wim,

I'm able to achieve a 100% File HDR hit rate with a 2 directories containing 1217 files and 1286. file header buffers (in pool #2).

So some operation that you're performing seems to clear/flush the caches...

Volker.

[Wim Van den Wyngaert]

Back on 5000.

Did a dir in a loop : also 100% (for 100 files).

Did 2 different dir in a loop : also 100% (200 files).

Did 2 dir of 10 directories each in a loop : 100% (2000 files)

Did 2 dir of 11.000 files each : 0%.

Reduced number of files to 3800. Dir in a loop : 100%. BTW : with cache 5 times faster.

Increased to 5700 files. Again 0%.

As soon as the number of files is bigger than the cache the cache isn't used ?

Wim

[Wim Van den Wyngaert]

It's probably FIFO. So, with 5000+ you simply create overhead of updating the cache but almost never benefit from it.

[Volker Halle · Accepted solution]

Wim,


BTW : with cache 5 times faster.


If you have a 100% hit rate from the File HDR cache, your CPU becomes the bottleneck, as all the headers are found in the cache and no disk IO is needed.

The buffer replacement algorithm for the buffer pools #0, #1 and #2 is: LRU = Leat Recently Used.

So yes, if the no. of files exceeds the no. of cache buffers, the file header cache won't help at all.

You can also flush the cache explicitly by e.g. using SET VOL/REBUILD disk

Volker.

[Wim Van den Wyngaert]

Case closed.

BTW : ana/sys display the % incorrect in clue mem/stat. Autogen however does it correctly.

Thanks Volker. Your the best.

Wim