topic Re: memory utilisation is 100% in Operating System - HP-UX

memory utilisation is 100%

jolight — Tue, 09 May 2006 15:43:58 GMT

Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.16 0.2% 0.0% 1.6% 98.2% 0.0% 0.0% 0.0% 0.0%
1 0.16 5.8% 0.0% 0.6% 93.6% 0.0% 0.0% 0.0% 0.0%
2 0.74 0.6% 0.0% 0.6% 98.8% 0.0% 0.0% 0.0% 0.0%
3 0.11 2.4% 0.0% 0.4% 97.2% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 0.29 2.2% 0.0% 0.8% 97.0% 0.0% 0.0% 0.0% 0.0%

Memory: 4804452K (2371208K) real, 6319612K (3121292K) virtual, 33096K free Page# 1/25

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
0 ? 0 root 127 20 32K 0K sleep 1763:29 1.90 1.90 swapper
1 ? 22853 patrol 154 20 13364K 6728K sleep 991:17 1.48 1.48 bgscollect
2 ? 39 root 152 20 7424K 7424K run 466:26 0.54 0.54 vxfsd
1 ? 6902 root 152 20 215M 4428K run 274:13 0.54 0.54 prm3d
0 pty/ttyp1 26090 root 178 20 7128K 5164K run 0:00 0.75 0.49 top
1 ? 6783 root -16 20 29076K 10532K run 407:15 0.35 0.35 midaemon

B3692A GlancePlus C.03.58.00 16:12:19 fetcher 9000/800 Current Avg High
-----------------------------------------------------------------------------------------------------------------------------
CPU Util S SA A | 7% 7% 7%
Disk Util V V | 4% 4% 4%
Mem Util S SU UB B |100% 100% 100%
Swap Util U UR R | 82% 82% 82%
-----------------------------------------------------------------------------------------------------------------------------
PROCESS LIST Users= 46
User CPU Util Cum Disk Thd
Process Name PID PPID Pri Name ( 400% max) CPU IO Rate RSS Cnt
--------------------------------------------------------------------------------
glance 26149 9056 168 root 17.8/10.2 0.2 5.7/21.1 6.4mb 1
swapper 0 0 127 root 1.4/ 1.5 105855 0.0/ 0.9 32kb 1
oracleDEPDE 574 1 154 oracle 1.4/ 0.0 14.5 0.0/ 0.0 12.5mb 1
ora_smon_DH 27969 1 156 oracle 0.0/ 0.3 18367.9 0.0/ 0.4 29.3mb 1
oracleDFSST 25964 1 154 oracle 0.0/ 0.0 0.1 0.0/ 0.0 40.6mb 1
oracleDFSST 25966 1 154 oracle 0.0/ 0.0 0.1 0.0/ 0.0 37.5mb 1

root:# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1024 0 1024 0% 0 - 1 /dev/vg00/lvol2
dev 1024 202 822 20% 0 - 0 /dev/uservg/swap1
dev 1024 201 823 20% 0 - 0 /dev/uservg/swap2
dev 1024 200 824 20% 0 - 0 /dev/uservg/swap3
dev 1024 201 823 20% 0 - 0 /dev/uservg/swap4
dev 1024 0 1024 0% 0 - 1 /dev/swdepotvg/swap5
dev 1024 0 1024 0% 0 - 1 /dev/swdepotvg/swap6
dev 1024 0 1024 0% 0 - 1 /dev/swdepotvg/swap7
dev 1024 0 1024 0% 0 - 1 /dev/swdepotvg/swap8
reserve - 6132 -6132
memory 6328 5770 558 91%
total 15544 12706 2838 82% - 0 -
root:#

What to do to reduce memory usage , swap space is enough or not?

Re: memory utilisation is 100%

Mel Burslan — Tue, 09 May 2006 15:53:37 GMT

Wondering about your kernel parameters dbc_max_pct and dbc_min_pct values.

could you please post the output of the following :

kmtune | grep dbc

Re: memory utilisation is 100%

Patrick Wallek — Tue, 09 May 2006 16:02:50 GMT

It appears that you have plenty of swap space.

You reduce memory usage by reducing the number of processes running, reducing the number of users or if using Oracle, you could reduce the size of your SGA.

I agree with Mel, I would look at dbc_max_pct and see what it is set to. If it is still the default value, 50, then it needs to be reduced. Although HP-UX is SUPPOSED to reduce your buffer cache, potentially down to the dbc_min_pct value, if the machine comes under memory pressure, which it appears you are.

Your real solution, if the kernel parameters are OK and you can't reduce users or reduce Oracle SGA (or something similar), is to add memory to the machine.

Re: memory utilisation is 100%

Ninad_1 — Tue, 09 May 2006 16:15:27 GMT

Hi,

Check the dbc_max and dbc_min kernel parameters as suggested.
You can check which are the top memory consuming processes by
UNIX95= ps -e -o "pid,user,sz,args" | sort -nr -k 3 | more
As per the output you have posted it seems that not many processes are running - so it seems as of now that a large chunk of memory is being allocated - like buffer, Oracle SGA or other shared memory segments rahther than any anything else, or there would be a runaway process - you can use the above command to see memory usage by processes.
Also chekc the shared memory usage-
ipcs -mb

Regards,
Ninad

Re: memory utilisation is 100%

Bill Hassell — Tue, 09 May 2006 16:18:20 GMT

Normally, using most of your memory is a good thing. But you are using way beyond your memory because swap is heavily occupied. If you look at the "m" screen in Glance, you will likely see a lot of page outs which seriously degrades performance. The m screen will also show the current buffer cache size (which should be about 500-1000 megs). You can:

1. stop running so many processes
2. reduce the size of SGA (but then Oracle will run slower)

But I imagine that neither choice is acceptable since the amount of work done will be reduced. The only choice is to add more RAM, about double what you have. Swapping (ie, paging) out to RAM is a sure sign that the system is too small for the amount of RAM. The top report shows swapper as the busiest process.

Re: memory utilisation is 100%

Steven E. Protter — Tue, 09 May 2006 16:32:47 GMT

Shalom,

I'd add more RAM.

You are utilizing swap pretty fully. Probably paging and system running slowly.

SEP

Re: memory utilisation is 100%

jolight — Wed, 10 May 2006 08:10:16 GMT

Hi all

The output of kmtune is as follows

# kmtune | grep dbc
dbc_max_pct 10 - 10
dbc_min_pct 5 - 5

Re: memory utilisation is 100%

jolight — Wed, 10 May 2006 09:33:02 GMT

Model: 9000/800/N4000-55
Main Memory: 8192 MB
Processors: 4
OS mode: 64 bit
# swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1048576 0 1048576 0% 0 - 1 /dev/vg00/lvol2
dev 1048576 215024 833552 21% 0 - 0 /dev/uservg/swap1
dev 1048576 214692 833884 20% 0 - 0 /dev/uservg/swap2
dev 1048576 213304 835272 20% 0 - 0 /dev/uservg/swap3
dev 1048576 213844 834732 20% 0 - 0 /dev/uservg/swap4
dev 1048576 0 1048576 0% 0 - 1 /dev/swdepotvg/swap5
dev 1048576 0 1048576 0% 0 - 1 /dev/swdepotvg/swap6
dev 1048576 0 1048576 0% 0 - 1 /dev/swdepotvg/swap7
dev 1048576 0 1048576 0% 0 - 1 /dev/swdepotvg/swap8
reserve - 6347676 -6347676
memory 6479828 5913664 566164 91%

Why there is difference in 8gb main memory and " memory 6479828 5913664 566164 91%" ?

Re: memory utilisation is 100%

Ninad_1 — Wed, 10 May 2006 09:47:31 GMT

>> Why there is difference in 8gb main memory and " memory 6479828 5913664 566164 91%" ?

They are refering to 2 different piece of information.
The 8192 MB is your physical memory (RAM)

and the memory you see in the output of swapinfo is the pseudo swap which is enabled using swapmem_on kernel parameter.
This parameter allows 75% of your physical memory to be thought of as swap - this is mainly useful when you have a huge memory and as swap needs to be reserved (not actually used) for every used memory, instead of adding swap space from disks which will be rarely used - this parameter enables pseudo swap.

Regards,
Ninad

Re: memory utilisation is 100%

jolight — Thu, 30 Aug 2007 00:02:04 GMT

Thank u all