24GB Buffer cache !!!

[jason_lee]

Hi,

Here is our setup, please advise if something wrong or can be improved:

1) HPUX: 11.23 (9000/800/SD32A)
2) Physical Memory : 80GB
3) Buffer Cache : 24GB (!!!!)
4) Bufpages : 6270336 (!!!!)
5) nbuf : 4495242
6) dbc_max_pct : 30
7) dbc_min_pct : 10

Server's purpose: Application server (mqm,tuxedo)

24GB of buffer cache!, do you think that it is a waste of resource(and money)?
How do we know how much of this 24GB is being utilized ?


jason.

[RAC_1]

Too much. Check it's usage with sar -v and keep adjusting. You should be getting write hit of at least 90+% to have effective use of that mush buffer cache. Also keeping it dynamic (use of dbc_max/min_pct) is better than keeping it static.

[melvyn burnard]

Ouch! That is a LOT of buffer cache.
From experience unless you really need it, reduce it. You could see performance issues caused because the system has to flush the caceh every so often, and that is a high priority process which can block others.
It will depened on what application you are using, and you could use top or Glance to see hopw much you may be using.

[Jaime Bolanos Rojas.]

Jason,

For each system the buffer cache setting is different, the best way to know which is the best for you is to tune the system:

Use sar -b command on check on your settings, you really want to see %rcache >= 90, and %wcache >=70. Keep on redusing the buffer cache until you get to that point.

If you have applications that use raw devices you might not get to see that buffer cache utilization.

Regards,

Jaime.

[Geoff Wild]

No reason to have 24GB of buffer cache!

For a good all around performance, you want budffer cache to be between 500 and 800 MB of ram.

I would set dbc_max_pct and dbc_min_pct to 1% each.

That would give more ram to your apps.

Rgds...Geoff

[jason_lee]

Thanks everyone for your prompt reply. Just a few more questions:

1) Are we using dynamic or static buffer cache now ? Bufpages = non-zero, suppose to be static, but the total "Buffer cache" is = 0.3 * 80GB = 24GB (dbc upper limit). Dynamic or static ?

2) We are not using raw device. All standard filesystems. Should we lower the dbc_max_pct ?

3) How can I find out if this (24GB) buffer cache is really causing problems to the processes in the system ? We still have 11GB of free memory (thank god !), there is no report of processes being swap out of memory.

jason.

[James R. Ferguson]

Hi Jason:

With regard to your last three questions:

1. I believe that you are really using a dynamic buffer cache since on 11.23 'bufpages' is deprecated. In fact, on releases after 11.23 the current buffer cache kernel parameters change again.

2. I agree with Geoff, lower your 'dbc_max_pct' and 'dbc_min_pct' to 1-2%.

3. The 'glance' tool is the best way to see your buffer cache. Two indicators of a large value causing adverse performance would be (a) memory pressure as evidenced by pageout activity (e.g using 'vmstat'); and (b) the 'syncer' daemon using large amounts of processor. 'Syncer' runs every 30-seconds to flush modified buffers from the buffer cache.

Regards!

...JRF...

[Jaime Bolanos Rojas.]

Jason,

The thing with buffer cache, if you have it too big, the you might cause some memory contention on the system.
Like you said you got plenty of free memory, but let's assume in the future you are going to consume more memory and for some reason at that moment you do not remember that you have a large buffer cache,
you will be wasting an incredible amount of memory never being used as buffer cache and that it's reserved for it.

Some people say that "hey memory is cheap" but not for me... :-)

Regards,

Jaime.

[Geoff Wild]

Yes - you are using dynamic as static was removed in 11.23.

A couple of threads on buffer cache:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=750342

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=749930

Rgds...Geoff

[Bill Hassell · Accepted solution]

The 'active' value for bufpages and nbuf is misleading. What counts is the compiled version of these parameters:

kctune -q bufpages nbuf

If these are both zero, then the buffer cache is indeed dynamic. The appropriate value for the DBC is min_dbc_pct=2 (the minimum allowewd) and max_dbc_pct=3 (which is still 2Gb based on 80Gb of RAM). Note that you may find a larger buffer cache (more than 1Gb) is actually useful in 11.23 (only) due to major rewrites in the handling of entries in the buffer cache.

The only way to reliably see buffer cache usage is to run Glance with the "d" window to look at logical (buffered) and physical reads and writes. Look at the second page in the d window for cache hit rates. Writes will normally have a lower hit rate.

[jason_lee]

Thanks everyone for advise/suggestions. I have a clearer idea on what should be done now.

[Zinky]

Jason,

I will check first with the vendor of your application if it requires that much. Remember, the rule with minimalistic buffer caches is mostly for a server that does Database serving and certain hybrid db/app serving.

Tuxedo and MQ may have certain "special" needs to merit a large buffer cache.


Just my few cents..

[jason_lee]

Nelson,

Thanks for the reply. I've checked the MQ & Tuxedo kernel requirements for HP-UX 11.X. Only changes to sem*,msg*,shmmax,maxuprocs,nfile etc (the usual one, just like for Oracle) nothing on buffer cache (nbuf,bufpages,dbc_*).


regards,
jason.

[jason_lee]

Thanks, closing this thread.