topic Re: Disk Buffer Cache in Operating System - HP-UX

Disk Buffer Cache

Dan Matlock_1 — Mon, 27 Mar 2006 13:21:31 GMT

Question: We have many servers that run Oracle and I want more disk buffer cache to handle disk I/O. Then I go into glance (tui) and use the 'm' option, I see one server at: Buf Cache: 7.8mb What kernel parameters affect this value? I assumed only dbc_max_pct and dbc_min_pct........

Re: Disk Buffer Cache

James R. Ferguson — Mon, 27 Mar 2006 13:25:31 GMT

Hi Dan:

Well, 'dbc_max_pct' and 'dbc_min_pct' control the UNIX buffer cache *if* 'nbuf' and 'bufpages' are *both* zero (0).

You can use 'kmtune -l' to examine your current settings.

See here for more details:

http://docs.hp.com/en/TKP-90202/TKP-90202.pdf

Regards!

...JRF...

Re: Disk Buffer Cache

Geoff Wild — Mon, 27 Mar 2006 13:27:18 GMT

Oracle has it's own buffer cache - you do not want to double buffer...

Mount indexes and data volumes like so:

delaylog,nodatainlog,largefiles,mincache=direct,convosync=direct

redo/logs

delaylog,nodatainlog

Rgds...Geoff

Re: Disk Buffer Cache

Dan Matlock_1 — Mon, 27 Mar 2006 13:33:27 GMT

thanx for updates, and I probably shouldn't have thrown the oracle reference in the mix... Most I/O is file disk read/writes outside of the oracle application.....

Re: Disk Buffer Cache

A. Clay Stephenson — Mon, 27 Mar 2006 13:34:16 GMT

7.8MB is much, much too small even if this is a pure Oracle server running raw/io.

The values dbc_min_pct and dbc_max_pct controll buffer cache iff bufpages = 0. If bufpages is set to a non-zero value (equal to the number of fixed 4KB pages) then it determines the buffer cache size. Even on a machine with little RAM (256MB); you would want the buffer cache to be at least 50MB or so; ideally on a machine with lots of memory, buffer cache on 11.11 should be in the range of 800-1600MB.

The other thing is that the dreaded "double-buffering" is really not so bad (in fact, it's good on HP-UX 11.11 and up). My performance tests indicate that Oracle actually performs best using fully cooked files and large SGA's on 11.11 and up. 11.0 and below worked best when the buffer cache was bypassed.

Re: Disk Buffer Cache

Geoff Wild — Mon, 27 Mar 2006 13:39:43 GMT

In that case, how much ram is in your system and what are the current settings for dbc_max_pct and dbc_min_pct?

As Clay says, around 800 MB works well...

So, if you have 10GB of ram, set dbc_max_pct to 8 to 10...try at 8, measure, then try at 9, measure, etc - and see what works the best...

Rgds...Geoff

Re: Disk Buffer Cache

Dan Matlock_1 — Mon, 27 Mar 2006 13:44:39 GMT

I have two systems (11.11) that I am comparing and both have about 6gb of memory and both are set 15-20 (one each)..... Strange that on that seems OK as bufpages at NBUF*2 which is 0 because NBUF=0.... The other which is 100% I/O is seet to NPROC*3 which means for us bufpages=1992!!!! Not sure why the difference for the same OS.
I am thinking I need to change the bufpages to NBUF*2 on the problem system.........?

Re: Disk Buffer Cache

A. Clay Stephenson — Mon, 27 Mar 2006 13:49:10 GMT

bufpages should have nothing to to do with nbuf (which should be at 0, the default). I would set set bufpages=204800 (800MB). I actually prefer fixed buffer caches to the dynamic versions.

Re: Disk Buffer Cache

Sandman! — Mon, 27 Mar 2006 13:58:25 GMT

Hi Dan,

Are you using raw or cooked volumes? Since you don't have to worry about the OS buffer cache if using raw volumes.

On the other hand for cooked volumes the problem of double buffering (one provided by the kernel and other by Oracle's SGA) comes into play. Application usage patterns of data will be the best guide for tuning this cache.

cheers!

Re: Disk Buffer Cache

Dan Matlock_1 — Mon, 27 Mar 2006 13:59:17 GMT

Thanks,,,, My focus is on "Buf Cache" in glance and this is a GOOD system:

Total VM : 4.66gb Sys Mem : 1.51gb User Mem: 4.00gb Phys Mem: 8.00gb
Active VM: 2.45gb Buf Cache: 1.20gb Free Mem: 1.30gb

This is a BAD system:
Total VM : 4.35gb Sys Mem : 1.01gb User Mem: 3.62gb Phys Mem: 7.97gb
Active VM: 3.03gb Buf Cache: 7.8mb Free Mem: 3.33gb

Just trying to increase the Disk Buffer Cache....... On the bad system bufpages is at 1992 (NPROC*3) and 7.8mb is certainly not db_max_pct of 20 out of 8gb...

The good system has Buf Cache @ 1.2gb and nbuf/bufpages is at 0 and dbc_max_pct=15 so that looks right if I have 8gb of memory....

Re: Disk Buffer Cache

Sandman! — Mon, 27 Mar 2006 14:11:04 GMT

Well your bad system has a static buffer cache i.e. 1992 * 4KB pages is roughly equal to 7.8MB and on your good system the buffer cache is dynamic owing to the nbuf = 0. Suggest you also look at buffer cache hit rates by looking at the output of sar on both systems, for ex:

# sar -b 5 50

and examine the averages at the end of the report for the values of %rcache and %wcache values.

cheers!

Re: Disk Buffer Cache

Geoff Wild — Mon, 27 Mar 2006 14:24:55 GMT

I wouldn't set buffpages manually anymore - HP seems to be limiting access to it (11.23 - you can't set from kcweb - but you can command line).

11.11

# kmtune |grep buf
bufcache_hash_locks 128 - 128
bufpages 0 - (NBUF*2)
nbuf 0 - 0
ndilbuffers 30 - 30
vol_maxstablebufsize 256 - 256
voliot_errbuf_dflt 16384 - 16384
voliot_iobuf_default 8192 - 8192
voliot_iobuf_limit 131072 - 131072
voliot_iobuf_max 65536 - 65536

11.23

# kctune |grep buf
physical_io_buffers 768 Default Auto
vol_maxstablebufsize 256 Default
voliot_errbuf_dflt 16384 Default
voliot_iobuf_default 8192 Default
voliot_iobuf_limit 131072 Default
voliot_iobuf_max 65536 Default
vxfs_bc_bufhwm 0 Default

But:

# kctune -q bufpages
Tunable Value Expression
bufpages 0 Default

So it is there - but HP wants you to use dbc...

Rgds...Geoff

Re: Disk Buffer Cache

Dan Matlock_1 — Mon, 27 Mar 2006 15:03:01 GMT

Thank you all for the feedback and I will make the change to allow dynamic allocation of buffer cache.....