topic Re: 90% average mem utilization 24x7 in Operating System - HP-UX

90% average mem utilization 24x7

Jeremy Loukinas_1 — Wed, 08 Jan 2003 20:35:45 GMT

I have a server running HPUX 10.20. Runs Informix and a couple other applications.

Here is my problem. My metrics are coming back every month with average memory utilization of 90% + at all times.

Can anyone give me some hints or suggestions on what to look at first?

Jeremy

Re: 90% average mem utilization 24x7

Sridhar Bhaskarla — Wed, 08 Jan 2003 20:39:51 GMT

Hi Jeremy,

Look for dbc_max_pct and dbc_min_pct values. By default, they are set to 50%.

sysdef |grep dbc

And the values you get are %'s of your physical memory. Decrease them to get around 300-400MB.

-Sri

Re: 90% average mem utilization 24x7

John Poff — Wed, 08 Jan 2003 20:40:31 GMT

Hi,

First thing I would look at is your buffer cache. What are your dbc_min_pct and dbc_max_pct kernel parameters set for? How much physical memory does the system have? If you have dbc_max_pct set to the default of 50%, you can drop it down to around 8-10% and pick up some extra memory.

JP

Re: 90% average mem utilization 24x7

Patrick Wallek — Wed, 08 Jan 2003 20:40:40 GMT

You're thinking 90% is bad? If it hovers around 90%, that is pretty good. The biggest indicator that you are having memory problems is if you start paging. Keep an eye on your 'swapinfo' output. If you never start paging, and your memory usage stays around 90%, then you have no problems.

I wouldn't really worry unless your memory usage goes up to and stays at 100%. As it is, you are using most of your RAM, which is a good thing.

Re: 90% average mem utilization 24x7

Ken Hubnik_2 — Wed, 08 Jan 2003 20:41:58 GMT

That's great!!! You are getting your moneys worth out of your memory. As long as you are not paging tremendously or having pages deactivated then you are fine.

Re: 90% average mem utilization 24x7

Jeremy Loukinas_1 — Wed, 08 Jan 2003 20:43:53 GMT

Here is my swapinfo output.

hp39:/ # swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 393216 28920 364296 7% 0 - 1 /dev/vg00/lvol2
reserve - 73264 -73264
memory 91656 64628 27028 71%
hp39:/ #

Re: 90% average mem utilization 24x7

Sridhar Bhaskarla — Wed, 08 Jan 2003 20:50:58 GMT

Hi,

Well, I will not be too excited if my memory hits above 90% on production systems as anytime I may get hit hard with extra load. I am happy as long as my memory util is around 80%.

Looks like you did swap a bit around 29MB of it is put on the device swap. It does tell that you are almost on the edge. But this is not a worrysome figure right now.

Check for dbc* parameters as indicated above and reduce them if you are using them with default values. You will get some memory freed up.

Also, look at your memory trend. If it gradually increasing over a period of time, then there is a chance that you have memory leaking processes. Generate regular outputs of

UNIX95= ps -e -o "vsz args" |sort
^ note the space

and see if the first column for any of the processes is growing over the time. That may be an indication of leaks.

-Sri

Re: 90% average mem utilization 24x7

Jeremy Loukinas_1 — Wed, 08 Jan 2003 20:51:51 GMT

Looks like I am using 7% of my swap.. Am I reading this correctly?
I formatted this in mb instead of kb.

hp39:/ # swapinfo -m
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 384 28 356 7% 0 - 1 /dev/vg00/lvol2
reserve - 71 -71
memory 90 64 26 71%
hp39:/ #

Re: 90% average mem utilization 24x7

Jeremy Loukinas_1 — Wed, 08 Jan 2003 20:56:03 GMT

what are the kernel params

dbc_min
dbc_max

?

They are set to the defaults of

5 and 50.

Jeremy

Re: 90% average mem utilization 24x7

Sridhar Bhaskarla — Wed, 08 Jan 2003 20:57:18 GMT

Oops my ^ shifted place after posting.

The space is after UNIX95=

-Sri

Re: 90% average mem utilization 24x7

Sridhar Bhaskarla — Wed, 08 Jan 2003 21:13:21 GMT

Hi Jeremy,

After looking at your swapinfo output, you do not seem to have more than 200MB of physical memory. So my initial statement to adjust dbc_max_pct to get 300MB would not be valid.

Make dbc_max_pct to 15% and dbc_min_pct 5% and see how your application works. That should release quite some memory.

-Sri

PS: dbc* are parameters for data buffer cache taken out of physical memory that sits in between user memory and the physical storage as a temporary storage. It buffers IO so the most recent reads do not have to go to the disk again and multiple write IOs can be combined if possible to reduce IO overhead.

Re: 90% average mem utilization 24x7

Frank Slootweg — Thu, 09 Jan 2003 12:31:06 GMT

*Where*, i.e. which utility, which parameter, etc., do you see "90% average mem utilization 24x7".

If it is really memory, i.e. RAM utilization, i.e. like "Mem Util" in Glance, then, as others indicate, 90% is *good*, not bad.

On HP-UX, main memory works like a cache and a good cache is always full. In order to allow fast startup of samll processes, HP-UX keeps memory nearly full, normally around 95%. If you see memory utilization of 97% and higher or/and paging/swapping, *then* memory pressure is to high.

Re: "swapinfo -m" output:

> Looks like I am using 7% of my swap.. Am I reading this correctly?

No, you should use the "-t" option for swapinfo and only look at the "total" line, *there* you will probably see a much higher USED percentage. The other lines are correct, but often only create confusion. See the manual page for details, but first take an asperin! :-)

Re: 90% average mem utilization 24x7

Pete Randall — Thu, 09 Jan 2003 12:44:57 GMT

Jeremy,

For an explanation of kernel memory parameters (dbc_min, dbc_max, etc.), see this thread:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x89a950dde50cd71190050090279cd0f9,00.html

Pete