Global swap nearly full

[Carlos Ruffin]

On a daily basis we are getting a critcal alert that global swap is nearly full. What can I do to stop this. This particular server is an app server for a SAP database. Listed below are the outputs from glance and 'swapinfo -t' :

CPU Util SSU U | 57% 57% 72%
Disk Util | 0% 4% 23%
Mem Util S SU UB B | 96% 96% 96%
Swap Util U UR R | 97% 97% 97%
--------------------------------------------------------------------------------
MEMORY REPORT Users= 1
Event Current Cumulative Current Rate Cum Rate High Rate
--------------------------------------------------------------------------------
Page Faults 2058 28144 395.7 329.1 983.2
Page In 555 8975 106.7 104.9 416.5
Page Out 0 12 0.0 0.1 1.5
KB Paged In 16kb 3.2mb 3.0 37.8 400.0
KB Paged Out 0kb 48kb 0.0 0.5 6.1
Reactivations 0 0 0.0 0.0 0.0
Deactivations 0 0 0.0 0.0 0.0
KB Deactivated 0kb 0kb 0.0 0.0 0.0
VM Reads 1 173 0.1 2.0 27.1
VM Writes 0 24 0.0 0.2 3.0
Total VM : 5.45gb Sys Mem : 223.5mb User Mem: 3.43gb Phys Mem: 4.00gb



hprdap05 /root # swapinfo -t
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192000 4449688 3742312 54% 0 - 1 /dev/vg00/priswap
reserve - 3742312 -3742312
memory 3217988 2382572 835416 74%
total 11409988 10574572 835416 93% - 0 -

[Jeff Machols]

really the only thing to do is add more swap. You can add device swap (logical volume) or filesystem swap. When you add device swap, you must reboot. If you need to add swap but can't boot, you could add filesystem swap until you have time to boot the machine, then add device swap.

[Uday_S_Ankolekar]

Hi,

Increase swap! and Also turn on psudo swap by setting swapmem_on to 1 in the kernel parms. This will rebuild the kernel and force a reboot

Look at this doc on performance Tuning..


http://us-support.external.hp.com/cki/bin/doc.pl/sid=48886897172ad0a691/screen=ckiDisplayDocument?docId=200000056493259

Good Luck,
-USA..

[Deshpande Prashant]

HI
Add more memory to system, if possible. Else add the device swap. You do not need to reboot machine for adding swap, it's only when you remove swap you need reboot machine.

Thanks.
Prashant.

[Sridhar Bhaskarla · Accepted solution]

Hi Carlos,

Looks like the spaces in the Mem Util section of your glance report got chopped so it is not clear how the utilization is. That gives a good indication. See if the memory utilization is due to buffer space. If you didn't change the default value of 50% of dbc_max_pct, that is too much and you may want to reduce it to get around 500MB. With 4GB of physical memory, you can set the value to 10-12%.

It looks like you are running low on memory. The used value is around 4GB on your device swap in swapinfo -t indicates it. If dbc_max_pct is not the culprit, you may want to add more memory. I wouldn't recommend to increase the swap too much as you already have enough (2xPhys Mem) with swap_mem on.

-Sri

[Carlos Ruffin]

Thanks all for you suggestions. My thinking was that we would have to add more memory or swap. The psuedo swap is already set to 1. I may need to go with Sridhar's suggestion. The hardest thing for me to do is to persuade the company to get more memory. Thanks again for your help.

[Mark Greene_1]

Until you get more memory installed, you can ask your SAP users to coordinate their efforts so that queries that run against the same files can be optimized with pre-selections of the data so you're not dragging different portions of the same file over and over through memory and swap.

--
mark

[Gordon Fong]

Hi all,

I do also encounter similar problems, but have some misundestanding, as show below, the pesudo-swap is already full, however, those device swap still has space, why it keep complain it's 99% full?? How it be calculated?

Bgds,
Gordon

--------------------------------------------------------------------------------
CPU Util S SN NARRU U | 56% 56% 56%
Disk Util F FV V | 36% 36% 36%
Mem Util S SU UB B | 98% 98% 98%
Swap Util U UR R | 99% 99% 99%
--------------------------------------------------------------------------------
SWAP SPACE Users= 16

Swap Device Type Avail Used Priority
--------------------------------------------------------------------------------
/dev/vg00/lvol2 device 4.0gb 2.4gb 1
/dev/vg00/swap2 device 8.0gb 2.4gb 1
pseudo-swap memory 12.4gb 12.1gb na







Swap Available: 25012m Swap Used: 17386m Swap Util (%): 99 Reserved: 24718m

[keith persons]

Gordon,

Your swap scenario is compounded by the high memory util in general. Psuedo-swap is really just a counter for swap with a fixed priority of 12. During the allocation phase the allocation appears to from the memory or psuedo-swap but when swap is actually used, all lower priority devices/files are consumed first. The 'reservation' from memory just speeds the allocation phase during process startup/growth.

Typically, psuedo-swap benefits degrade as memory utilization increases. With memory utilization values in excess of 80% sustained, psuedo-swap will more often hinder throughput.

With the memory util values you posted, I would suggest disabling psuedo-swap (set swapmem_on in kernel to 0) and add more swap devices.

keith

[Pete Randall]

Gordon,

For a number of reasons you are better off starting a new thread for your question, perhaps referring back to this question. You'll get better response and you'll be able to assign points so that future viewers are better able to glean what worked for you from what didn't.

Pete