hp-ux swap utilization so high >90%

[Ganesan R]

Hi Joseph,

>>>>
swap has 40G and used approx 11G
so ~it should reserve 29G
why in this case it juse reserve 3729M ??
<<<<<<

Don explanation is well enough to understand how swap system works.

In simple terms, whenever a process starts some amount of swap space will be reserved. Note that only "reserved" not "used". And the "some amount" represents the amount of swap space needed to move the process to swap area in case the physical memory is full. So when the reservation reaches to the maximum available swap then you will not be able to new process. In this situation only you have to increase the swap.

If "USED" column on dev/fs is more, it indicates there is a physical memory pressure so system started using swap area for swap in/out activity. In that case you have to go far increasing physical memory.

You can read this also to understand the swap memory usage..
http://docs.google.com/View?docid=dc7s6mzr_56cgsg9g

[joseph123]

hi guys

[root@SAPDBCI]/ # swapinfo -atm
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 20480 230 20250 1% 0 - 1 /dev/vg00/lvol2
dev 10240 230 10010 2% 0 - 1 /dev/vg00/lvol9
reserve - 30261 -30261
memory 40925 36278 4647 89%
total 71645 66999 4646 94% - 0 -
=========================
this server is start to used swap
is it possible to find out why memory used (36278) is so high?
or identify which process used many memory ?

[Dennis Handly]

>this server is start to used swap

Are you looking at the vmstat output for "po"?

>is it possible to find out why memory used (36278) is so high? or identify which process used many memory?

Because you are running too many processes that want to use lots of swap. Or you have configured SAP to use too much.

You can look at the memory used in the ps(1) output. You can also look at "ipcs -ma" for shared memory segments.

[joseph123]

hi guys

i met a problem
from database listener.log
i did not find any other alert from other log.

11-SEP-2009 04:14:51 * (CONNECT_DATA=(SID=PR1)(GLOBAL_NAME=PR1.WORLD)(CID=(PROGRAM=C:\usr\sap\PR1\D05\exe\disp+work.EXE)(HOST=PRDAP5
)(USER=pr1adm))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.158.4.16)(PORT=2302)) * establish * PR1 * 12500
TNS-12500: TNS:listener failed to start a dedicated server process
TNS-12540: TNS:internal limit restriction exceeded
TNS-12560: TNS:protocol adapter error
TNS-00510: Internal limit restriction exceeded
HPUX Error: 12: Not enough space

==============================
seems like i should modify two thing
1. Modify kernel parameter
2. add swap

with regards 1, i have checked shmmax, this paremeter has been set to the size of the RAM which is 42 GB.
and maxdsiz*, the kernel parameter has been set as per SAP recommendation.
so, i don't think i sould modify any kernel parameter.

so, i read some data from http://docs.google.com/View?docid=dc7s6mzr_56cgsg9g

i suspect i should add swap

but, from our curreut status

# vmstat
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
5 7 0 12523101 633358 244 38 2 0 0 0 2 11322 67967 6615 21 17 62


# swapinfo -atm
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 20480 862 19618 4% 0 - 1 /dev/vg00/lvol2
dev 10240 861 9379 8% 0 - 1 /dev/vg00/lvol9
reserve - 28996 -28996
memory 40925 36228 4697 89%
total 71645 66947 4698 93% - 0 -

we can see swap just reserve not used, why does our swap not enough?
please give me some advise, thanks

[Dennis Handly]

>maxdsiz*, the kernel parameter has been set as per SAP recommendation.

What are the values?

>total 71645 66947 4698 93%

You only have 4.6 Gb free to allocate. Did you do swapinfo at the time of failure?
How much swap that new process going to use?

>we can see swap just reserve not used ...

That's what reserved mean, it can't be used for anything else.

[joseph123]

Shalom,

kernel parameter Current value
maxdsiz 1073741824 1073741824 yes
maxdsiz_64bit 17179869184 17179869184 yes
shmmax 42949672960 42949672960 yes

i did not do swapinfo at that monment, i'm not sure how much swap that the new process will going used.

[Dennis Handly]

>I did not do swapinfo at that moment, I'm not sure how much swap that the new process will going used.

(Did you ever mention which OS version?)

Then you could be out of swap. You could use size(1) on the executable and all shlibs to see how much you need at the start.
Of course it could die latter on a big malloc/mmap.

[joseph123]

the os version is 11iv3 B.11.23

[joseph123]

# swapinfo -atm
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 20480 830 19650 4% 0 - 1 /dev/vg00/lvol2
dev 10240 828 9412 8% 0 - 1 /dev/vg00/lvol9
reserve - 29062 -29062
memory 40925 35720 5205 87%
total 71645 66440 5205 93% - 0 -

if i want to create a new swap about 10G
our pe size is 16M
i should type
# lvcreate -l 640 /dev/vg00
if new lv named lvol10
i want enable swap via /etc/fstab

# more /etc/fstab
/dev/vg00/lvol3 / vxfs delaylog 0 1
/dev/vg00/lvol1 /stand hfs defaults 0 1
/dev/vg00/lvol4 /opt vxfs delaylog 0 2
/dev/vg00/lvol5 /tmp vxfs delaylog 0 2
/dev/vg00/lvol6 /usr vxfs delaylog 0 2
/dev/vg00/lvol7 /var vxfs delaylog 0 2
/dev/vg00/lvol8 /home vxfs delaylog 0 2
/dev/vg00/lvol9 ... swap pri=1 0 0
devap1:/export/archive/ERP /usr/sap/archive/ERP nfs rw,suid 0 0

i should edit /etc/fstab
add a swap entry
/dev/vg00/lvol10 ... swap pri=1 0 0

and
# swapon -a

that's all, rite?
i don't need to stop application server which is on the server, rite?
i also don't need restart server, rite?