Troubles with memory and swap

[JMI]

Troubles with memory and swap...

machine: HP - L3000
3 cpus
4 gb

OS: hpux 11.11
...SAP CRM and ORACLE 8.1.7.4.0

When we start sap, swap is empty, but some hours later, when users have been working with the application, swap is increased more and more,...and finally, this is the situation:

# swapinfo -mt
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 4084 12 100% 0 - 1 /dev/vg00/lvol2
dev 12504 12472 32 100% 0 - 1 /dev/vg00/extraswap
dev 4000 1256 2744 31% 0 - 2 /dev/vg00/extraswap2
reserve - 2789 -2789
memory 3088 3066 22 99%
total 23688 23667 21 100% - 0 -

When we there aren??t processes running in sap, the amount of swap is the same.
If we have a look to glance application, oracle use less than 1 gb and sap about 2 gb...

B3692A GlancePlus C.03.35.00 08:59:15 fsacrm02 9000/800 Current Avg High
--------------------------------------------------------------------------------
CPU Util S SU U | 87% 70% 87%
Disk Util FFV V | 53% 53% 98%
Mem Util S SU UB B |100% 100% 100%
Swap Util U UR R |100% 100% 100%
--------------------------------------------------------------------------------
APPLICATION LIST Users= 1
Num Active CPU AvgCPU Logl Phys Res Virt
Idx Application Procs Procs Util Util IO IO Mem Mem
--------------------------------------------------------------------------------
1 other 20 4 2.0 1.0 160.0 57.8 290.7mb 1.05gb
2 network 32 3 0.0 0.0 1.0 0.0 12.5mb 40.3mb
3 memory_management 5 3 4.3 4.2 0.0 85.8 2.1mb 2.5mb
4 other_user_root 69 14 0.7 0.9 0.8 0.3 48.5mb 134.9mb
5 CRP Oracle processe 1 1 0.0 0.0 0.0 0.0 1.7mb 5.5mb
6 CRP Oracle shadow p 29 9 12.8 8.6 332.0 92.6 735.1mb 993.4mb
7 CRP R/3 disp+work 30 13 65.1 54.0 4.6 88.0 2.16gb 21.3gb
8 CRP R/3 processes 3 2 1.5 0.9 0.3 0.0 14.2mb 47.3mb

We have problems with backup and of course with sap. We can??t run online backup - omniback,...

[Critical] From: BDA-NET@fsacrm02 "CRP" Time: 11/07/2003 0:01:26
[80:1003] Cannot allocate/attach shared memory (IPC Cannot Create Shared Memory Segment
System error: [12] Not enough space
) => aborting.

[Normal] From: OB2BAR@fsacrm02 "CRP" Time: 11/07/2003 0:01:26
Starting OB2BAR Backup: 05 fsacrm02:CRP.3 // SAP

[Critical] From: OB2BAR@fsacrm02 "CRP" Time: 07/11/03 00:01:26 [80:1011] Cannot fork NETIO process ([12] Not enough space) => aborting.
[Critical] From: OB2BAR@fsacrm02 "CRP" Time: 11/07/2003 0:01:26
[80:1011] Cannot fork NETIO process ([12] Not enough space) => aborting.

Maybe we need more physical memory, but we don??t know why consume that wide amount of swap,...and we don??t know why don??t release swap when processes aren??t running...

Could you help us?
Thanks in advance.

[Stefan Farrelly]

Yes, youre completely out of memory!

You have 4GB of RAM and almost 18GB of device swap used - which means you are about 18GB short of RAM! Ive never seen a server so bad (so short of RAM).

Swapinfo doesnt update when swap is freed up by stopping processes so dont be concerned that swapinfo still shows very high device swap usage even though you might have stopped all your databases. Use something more sophisticated like glance, it will update once you shutdown processes. The only way to reset swapinfo properly is to reboot your server.

You clearly need to work out what is using so much memory (4GB RAM + almost 18GB of device swap). Use this command to try to id which processes are using what (highest memory hoggers listed first);

UNIX95= ps -e -o vsz=Kbytes -o ruser -o pid,args=Command-Line | sort -rnk1 | more

But in my opinion your so short of RAM only a large RAM upgrade will solve your problems. Your performance must be terrible with so much swapped out to disk!

[Steve Steel]

Hi


You need more something but try

swapmem_on on 0 in the kernel and free up your memory.

Oracle does its own buffer caching yet you have memory caching on.This leaves very little memory free for applications and makes more swap.

A lot more memory or swapmem_on 0 in the kernel and add swap .

The swapmem_on option is easier to test


Steve Steel

[Steve Steel]

Hi

changed my mind

put dbc_min_pct dbc_max_pct both on 10
in the kernel and make sure you have 20GIG
disk swap available


Steve Steel

[RAC_1]

setting swapmem_on to 0 will free up some memory.

Also check buffer cache and make sure they are not unneceesarlity high. 5 to 10 % should be Ok, depending upon your oracle load.

But I doubt you running sap which seems heavyly used. So all you can do is add some RAM that will improve the performance. OR else even with 20gb of swap space you wont get any noticable performance improvement.