filecache_max at 64%

[Lisa Dingman]

Our filecache_max is set to the default(16.3G or 32G), 50% but shows 96% usage. We have recently updated our licenses for our database product and made some changes to the udtconfig file for that. Not sure if this is the result of that or has been at 96% always. I will be able to reboot tonight to see if that makes a difference but could someone tell me what this is indicative of and what could be the possible problems with this being at 96% usage. Thank you in advance.

[Vivek Bhatia]

Hi Lisa,

Which version of HPUX?

How are you checking the settings of filecache_max? -Please paste the output in this forum

How are you checking the utilization of filcache_max ?

Regards
Vivek Bhatia

[James R. Ferguson · Accepted solution]

Hi Lisa:

I thing you mean that the filecache is at 96% of the 50% of physical memory that you apportioned (by default).

The 50% ceilig may or may not work well in your environment. As always, your mileage may vary. Do not equate the newer filecache parameters with the older buffer cache ones ('dbc_(min|max)_pct).

http://docs.hp.com/en/B2355-60130/filecache_max.5.html

Regards!

...JRF...

[Vivek Bhatia]

Hi,

Paste the output of kcusage and we might be able to understand the situation better.

Regards,
Vivek

[Steven E. Protter]

Shalom Lisa,

On any active system the filecache is going to get used, no matter the size.

The fact that its not 100% used, means its probably big enough.

If you reboot your system and check it after boot, you will find usage is lower than it is now.

This does not in itself indicate problems.

I am a member of the Bill Hassell school of administration. Problems get worked on when three is evidence, user complaints, slow response and such. I do many pro-active things like patching, but I don't get excited by a statistic like this.

SEP

[Lisa Dingman]

We have an integriy rx6600, hpux 11 iv3. Yes you are correct it shows 96% of the 50% of filecache_max. Just happened to see on graph in SMH.
Thank you for the advice about kcusage, am a programmer who is now is sysadmin so is new to me. This looks like it happened about the same time that the new udtconfig file for our database was updated, however they only suggested changing minimal amounts on fields, I have attached the changes requested.
# kcusage -d filecache_max
Tunable: filecache_max
Setting: 16322822144
Time Usage %
=============================================
Tue 06/23/09 11:00 CDT 4336123904 26.6
Tue 06/23/09 12:00 CDT 4386189312 26.9
Tue 06/23/09 13:00 CDT 4665380864 28.6
Tue 06/23/09 14:00 CDT 4702359552 28.8
Tue 06/23/09 15:00 CDT 4587450368 28.1
Tue 06/23/09 16:00 CDT 4661751808 28.6
Tue 06/23/09 17:00 CDT 4680110080 28.7
Tue 06/23/09 18:00 CDT 4686954496 28.7
Tue 06/23/09 19:00 CDT 4694245376 28.8
Tue 06/23/09 20:00 CDT 4700508160 28.8
Tue 06/23/09 21:00 CDT 4714946560 28.9
Tue 06/23/09 22:00 CDT 4811407360 29.5
Tue 06/23/09 23:00 CDT 4861390848 29.8
Wed 06/24/09 00:00 CDT 4864335872 29.8
Wed 06/24/09 01:00 CDT 4893179904 30.0
Wed 06/24/09 02:00 CDT 4827762688 29.6
Wed 06/24/09 03:00 CDT 16319885312 100.0
Wed 06/24/09 04:00 CDT 15383179264 94.2
Wed 06/24/09 05:00 CDT 15420329984 94.5
Wed 06/24/09 06:00 CDT 15306989568 93.8
Wed 06/24/09 07:00 CDT 15415889920 94.4
Wed 06/24/09 08:00 CDT 15671930880 96.0
Wed 06/24/09 09:00 CDT 15765381120 96.6
Wed 06/24/09 10:00 CDT 15817781248 96.9
#
I tried jus to see if anything changed to change back to the previous udtconfig file (for our datatel database) to see if that would make a difference because that is the only thing I can see that changed but so far it didn't:
kcusage -h filecache_max
Tunable: filecache_max
Setting: 16322822144
Time Usage %
=============================================
Wed 06/24/09 10:05 CDT 15784185856 96.7
Wed 06/24/09 10:10 CDT 15792558080 96.8
Wed 06/24/09 10:15 CDT 15793577984 96.8
Wed 06/24/09 10:20 CDT 15798099968 96.8
Wed 06/24/09 10:25 CDT 15808950272 96.9
Wed 06/24/09 10:30 CDT 15802822656 96.8
Wed 06/24/09 10:35 CDT 15811796992 96.9
Wed 06/24/09 10:40 CDT 15817834496 96.9
Wed 06/24/09 10:45 CDT 15831109632 97.0
Wed 06/24/09 10:50 CDT 15839285248 97.0
Wed 06/24/09 10:55 CDT 15839088640 97.0
Wed 06/24/09 11:00 CDT 15836065792 97.0

[Vivek Bhatia]

Hi Lisa,

How much is the physical Ram in your machine?

1. SHMMAX: 1073741824 1073741824 -This is currently 1GB and it should be equal to Physical RAM size.

Was there any changes on the server between 23/24?

Thanks
Vivek Bhatia

[Lisa Dingman]

We have 32G of RAM. We have web daemon processes that are allocated given amounts of memory so am not sure what upgrading this to 16G would do. The only change was that I saved the new udtconfig file, but that wouldn't take effect until the Database was restarted (4 AM this morning). I did load DataProtector on someone's pc yesterday at this time but I don't see how/if this would have effected this, maybe someone more knowlegeable than I see's a connection. That is the only thing that happened and it was exactly about 2 pm yesterday when the amount started climbing.

[Lisa Dingman]

Forgot to add, is there any problem with increasing shmmax? The man pages says to raise if it is below the max and user program are attempthing shmget segments ... and get EINVAL error message.(not happened) It shows 3% usage of the default of 1073741824, would that say the value is adequate since only at 3% usage?

[Vivek Bhatia]

Hi Lisa,

Well it is always recommended to have SHMMAX to be set equal to the Physical RAM.

you can run a kcusage to check how much this kernel parameter is in use?

Regards,
Vivek

[James R. Ferguson]

Hi Lisa:

> Forgot to add, is there any problem with increasing shmmax? The man pages says to raise if it is below the max and user program are attempthing shmget segments ... and get EINVAL error message.(not happened) It shows 3% usage of the default of 1073741824, would that say the value is adequate since only at 3% usage?

Why do you want to diddle with the current setting? You indicate that you are NOT getting EINVAL errors for 'shmget()' memory segment acquisitions. I'd leave this alone.

Regards!

...JRF...

[Lisa Dingman]

Yes, this only shows 3.1% usage for shmmax. Now filecache_max is at 100%. Could this somehow be related to a DataProtector install. That is the only thing that changed when filecache_max increased yesterday.

# kcusage -d shmmax
Tunable: shmmax
Setting: 1073741824
Time Usage %
=============================================
Tue 06/23/09 15:00 CDT 33554432 3.1
Tue 06/23/09 16:00 CDT 33554432 3.1
Tue 06/23/09 17:00 CDT 33554432 3.1
Tue 06/23/09 18:00 CDT 33554432 3.1
Tue 06/23/09 19:00 CDT 33554432 3.1
Tue 06/23/09 20:00 CDT 33554432 3.1
Tue 06/23/09 21:00 CDT 33554432 3.1
Tue 06/23/09 22:00 CDT 33554432 3.1
Tue 06/23/09 23:00 CDT 33554432 3.1
Wed 06/24/09 00:00 CDT 33554432 3.1
Wed 06/24/09 01:00 CDT 33554432 3.1
Wed 06/24/09 02:00 CDT 33554432 3.1
Wed 06/24/09 03:00 CDT 10469376 1.0
Wed 06/24/09 04:00 CDT 33554432 3.1
Wed 06/24/09 05:00 CDT 33554432 3.1
Wed 06/24/09 06:00 CDT 33554432 3.1
Wed 06/24/09 07:00 CDT 33554432 3.1
Wed 06/24/09 08:00 CDT 33554432 3.1
Wed 06/24/09 09:00 CDT 33554432 3.1
Wed 06/24/09 10:00 CDT 33554432 3.1
Wed 06/24/09 11:00 CDT 33554432 3.1
Wed 06/24/09 12:00 CDT 33554432 3.1
Wed 06/24/09 13:00 CDT 33554432 3.1
Wed 06/24/09 14:00 CDT 33554432 3.1
#

[Don Morris_1]

The system is _supposed_ to cache between filecache_min and filecache_max as needed and accounting for overall system pressure. There is no problem with that -- you should expect filecache usage near 100% in many cases. (Especially when filecache_max is low. Your system is not low... but obviously, there's some significant file traffic moving around that caused the cache to fill up. If that was of the "read (or write) once, never reference again" then as soon as there's pressure... those pages will be the first to go!)

Is there no free memory that you believe the filecache_max represents an incorrect use of RAM?

[Don Morris_1]

Oh, and if you're really paranoid about this -- you don't need to reboot. Set filecache_max to filecache_min (be prepared to see vhand taking cycles) to force the cache to shrink, then revert to the original value if desired.

If you don't believe your filecache should be using that amount of memory in the first place, just set the tunable lower and move on.

[Lisa Dingman]

I think my main issue is trying to find out what made this jump at 2pm yesterday the exact same time as the install of DataProtector. If someone had incorrectly performed the install and tried to install a new cell manager instead of a client install of DataProtector, is that something I could check for. In troubleshooting anything you wonder what was the last thing that happened before the change in 'event' and it could just be a coincidence but seems very odd. I have to reboot tonight anyway for the few kernal changes for our added licenses for our Database so will see what that does. Thank you, I appreciate all your help.

[Lisa Dingman]

Oh my goodness, my apologies to everyone. The change was at 2AM not 2PM and that was when the Database was stopped and restarted to accept the new udtconfig perameters. I cannot believe I didn't see this. SO hopefully when I reboot the system at 3am tommorrow everything shoud be in sync. I guess this didn't seem obvious as the perameters didn't change much as far as a few percent. Hopefully I can report good news tommorrow.

[Lisa Dingman]

Thank you everyone, what a dummy, I just didn't pay attention to the time and connected 02:00 with 2 pm which took me on a wild goose chase. I did reboot and the filecache_max peram is back down to 2%. I learned alot though, have had no training as sys admin wasn't aware of kcusage so a big thanks to all who helped!!Learned alot through this.

[P.J. Cruz]

Lisa we are also a Unidata 7.1.11 customer what is your version? my email is pj.cruz@rcc.edu

[Hein van den Heuvel]

P.J Popped this topic to the top, where I saw it for the first time.

SEP wrote>> The fact that its not 100% used, means its probably big enough.

I would go one step further.
I would argue that the cache is clearly TOO BIG in a general sense, or too big for the available physical memory.

In either case there is no reason to increase, and little or no reason to decrease.

For me a healthy cache setting would fill to 100% and then start to flush out old data to make room for new data. If it does not fill, then it is too big, but that does not really cost anything. Just memory gone unused, which perhaps you can tell a database or such to use.
If it wanted that 100% but was trimmed down to 96% because of more urgent memory pressure, then so bit it. It relinquished the memory like it should. If that is an ongoing struggle, then I suppose you could trim the _MAX some to make it self-limiting/pro-active versus re-active. No big deal... IMHO

Lisa wrote (@19:38:38 ) .. " when filecache_max increased yesterday."
I'm kinda curious what made you increase it considering the replies and explanations earlier.

Lisa wrote (@Jun 25, 2009 14:39:42)
GMT N/A: Question Author
"I did reboot and the filecache_max peram is back down to 2%"

Really? Are you referring to actual use, relatively shortly after a reboot, or was the max actually set to 2% ?!. That would seem very wrong. HPUX 11v3 is very happy and capable to manage much more aggressive cache settings.

A max of 2% would likely be holding back the system, forcing IO's which could be avoided.


fwiw,
Hein van den Heuvel
HvdH Performance Consulting