topic Re: Understanding Memory Usage in Operating System - HP-UX

Understanding Memory Usage

Ken Polakowski_1 — Thu, 18 Dec 2003 06:47:25 GMT

Hi:

I'm trying to understand memory usage on one of my systems. I've attached the lines from the character cell glance memory report of glance which show me that I have 8GB of physical memory but only shows me 4GB of Virtual memory. Where are the ther 4GB of physical memory?

Thanks in Advance,

Ken Polakowski

Re: Understanding Memory Usage

Jean-Luc Oudart — Thu, 18 Dec 2003 06:57:09 GMT

Ken,

The total private virtual memory (in
KB unless otherwise specified) at
the end of the interval. This is
the sum of the virtual allocation of
private data and stack regions for
all processes.

from glance

regards,
Jean-Luc

Re: Understanding Memory Usage

Ken Polakowski_1 — Thu, 18 Dec 2003 07:36:37 GMT

Much Thanks for the quick reply. Its early and I've been swamped with stuff. I'm fairly new at HP-UX and I had a DBA panic last night when he looked at the memory statistics from top and came to the wrong conclusion that the system was only seeing 4GB memory raised the question with the director of MIS, who in turn wanted me to immeidately call HP field service and get the memory replaced. I should have stepped back, taken my normal deep breath and analyzed the situation. Glance really shows me my memory usage as:

System Memory: 1.53GB
Buf Cache: 1.60GB
User Mem: 3.05GB
Free Mem: 1.83GB

The last time I checked that added to 8.01GB.

Re: Understanding Memory Usage

Bill Hassell — Thu, 18 Dec 2003 08:15:00 GMT

Rule of thumb: ignore what top says about memory. It has always been incomplete and very misleading for today's virtual memory Unix systems. Another note: 1.6Gb for your buffer cache is more than double the recommended maximum size. Keep your buffer cache to about 500 megs (for *Gb of RAM, dbc_max_pct=5, dbc_min_pct=2) and tell your DBA that there's about 3Gb of additional memory available for applications like Oracle. Note that there are severe restrictions for large memory (Gbytes) applications that are old 32bit versions.

Re: Understanding Memory Usage

Jean-Luc Oudart — Thu, 18 Dec 2003 08:22:41 GMT

Ken,

Just ensure you're not paging out (swap space option in glance).
Buffer cache is too high and you could save some memory in this area.

Don't forget to assign points to people and give appreciation to their response.
Furthermore, a "bunny" will tell people when a question was answered and solve the problem.

regards,
Jean-Luc

Re: Understanding Memory Usage

Ken Polakowski_1 — Thu, 18 Dec 2003 08:30:42 GMT

I was here when he was experiencing problems during a very large import. Memory usage was approximately 85%. One thing that I noted was that swapinfo showed that my entire 4GB swap space was reserved (-4194303). It also showed the usage on the swap device at zero.

As for the dbc_min and dbc_max. I was here when the systems were installed and they were both set by an HP person to 20%. This effectively disables the dynamic buffer caching (right?). Was this a good idea? How can I monitor the cache usage so that I can tweak these numbers appropriately?

Thanx

Re: Understanding Memory Usage

Ken Polakowski_1 — Thu, 18 Dec 2003 08:37:03 GMT

Oops! forgot this. When the import was running. I did have trouble starting applications due to memory unavailable problems.

Anything that I should be aware of when he runs another one of these processes.

There is nothing other then Service Guard and this Oracle instance running on this L class machine.

Re: Understanding Memory Usage

Jean-Luc Oudart — Thu, 18 Dec 2003 08:40:12 GMT

Ken,

I attached the Unix performance cookbook.
Read the paragraph on buffer pages (bufpages).
From waht is said on this forum, it quite depends on which version of HPUX you're running (HPUX11.0 or HPUX11i).
If you run HPUX11.0 300-400Mb should suffice fro the job,

regards,
Jean-Luc

Re: Understanding Memory Usage

Jeff Schussele — Thu, 18 Dec 2003 08:48:37 GMT

Hi Ken,

If swapinfo showed ALL the swap space reserved then you need to add some more swap space. Just add it as secondary swap. You must have enough swap space to accomodate all processes that will be run concurrently or the process will not be allowed to run.

Rgds,
Jeff

Re: Understanding Memory Usage

John Carr_2 — Thu, 18 Dec 2003 08:57:07 GMT

Hi Ken

I always check for amount of physical memory with following command, just another bit of hpux for you.

grep -i phy /var/adm/syslog/syslog.log

:-)

Re: Understanding Memory Usage

Bill Hassell — Thu, 18 Dec 2003 09:27:17 GMT

The buffer cache is not disabled by a certain percentage. The default in HP-UX is 5% and 50% (which is WAY too large for an 8Gb machine). Setting dbc_max_pct to 5% will limit the maximum buffer cache to about 500 megs which is fine. To eliminate the buffer cache feature on a specific mountpoint, you need the optional Online JFS product which will give you new options:

convosync=direct,mincache=direct,nodatainlog

which should be applied only to data directories (not indexes, archive logs, executable directories, etc). Reserved swap is simply mapped but nothing has actually been moved to the swap area. Your only concern as far as swap space is concerned is the page out rate (ignore page in). If it is in the 0-10 range, all is well. over 20 for long periods means you need a bit more RAM, over 100 for long periods means you probably need 50% more RAM to improve performance.

As far as problems with the large import, there have been numerous discussions about Oracle and memory usage--most of the issues are with 32bit programs and the shared memory area. Running out of memory very seldom means that there is not enough virtual memory, it often means that you've hit a kernel safety limit such as maxdsiz (which can be changed).

You can monitor the buffer cache size in Glance. If it stays near the dbc_max_pct value then there is no pressure from applications to require more RAM. The cache is always busy and after a few hours following a reboot, virtually all of the cache will be used. This is by design. The cache will shrink in size if processes need more RAM and you'll see this in Glance. Having more RAM available will not help Oracle until the DBA changes local and shared memory values in the Oracle config files.

Re: Understanding Memory Usage

Ken Polakowski_1 — Thu, 18 Dec 2003 10:23:35 GMT

Thanx for the cookbook. I plan to set aside some time today/this evening to go through it. I'll plan on adding some secondary swap.

Questions about the Min and max parameter:

What effect does setting them equal have? I've been watching the activity this morning and the value never changes.

Should I adjust them say to 5% and 10% and let them float?

Unfortunately, These systems are in a validated state and I need to present justification for making any changes to our change control committee for approval.

I also just ordered the Sauers/Weygant book which I should have in tme to keep me busy for this weekend.
Thanx again,

Ken

Re: Understanding Memory Usage

Bill Hassell — Thu, 18 Dec 2003 14:07:07 GMT

Making them equal means that the amount allocated will not shrink down in case large amounts of memory are needed. Instead, once physical memory is almost exhausted, some processes will be deactivated and then portions of them will be moved (paged) to the swap area. The purpose of a dynamic buffer cache is to shrink without having to stop processes and swap them out. Two very useful docu,ments for your management and DBA are found in /usr/share/doc: mem_mgt and proc_mgmt. Print or email copies for the curious. A related document (for 32 bit programs) is mem_wndws.txt. If you are running 11.11, this directory was gutted for unknown reasons but you can find them all in 11.00 systems or the memory paper at docs.hp.com:

memory_management: http://docs.hp.com/cgi-bin/otsearch/getfile?id=/hpux/onlinedocs/1218/mem_mgt.html&searchterms=pde%7cpdir&queryid=20031218-120015

I will attach proc_mgmt to this message and mem_wndws to the next message.

Re: Understanding Memory Usage

Bill Hassell — Thu, 18 Dec 2003 14:08:22 GMT

Here is the memory windows white paper from 11.00 (applicable to 11i). NOTE: you need to be up to date on patches for 11.00 for memory windows to work.

Re: Understanding Memory Usage

Tim Sanko — Thu, 18 Dec 2003 15:09:47 GMT

Ken,

I let mine float from 5%-10% min-max.

I would probably reduce it on an 8 GB ram Machine. I do get 98% cachs hits, (lucky me).

I would look at the DB parms to see how large SGA and buffers are. I am also a DBA and have found that if they change parms we can go from a running state to a swap condition quickly...

Tim