Probably easy swapinfo question

[Simon Hargrave]

On of our servers has recently complained of running out of memory. This is the current swapinfo -tam output : -

Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 1635 2461 40% 0 - 1 /dev/vg00/lvol2
reserve - 2461 -2461
memory 3126 2715 411 87%
total 7222 6811 411 94% - 0 -

at the time of the problem, memory and total were 100%, and dev was 42%

So in terms of "virtual memory", I have 4Gb device, plus 3Gb (75% of RAM) as pseudo-swap, so should be able to allocate 7Gb (as the total suggests).

So why, even though there's still nearly 60% free device is it allocating no more? Shouldn't the reserve take up the dev space available? And will adding more dev swap help here (obviously more RAM is better, but not currently possible...)

Ta,


Sy

[James Murtagh]

Hi Simon,

In the kernel there is a counter set which is basically the total from your device and psuedo swap areas. Any further requests will be checked against this counter and will fail if your total free count is not sufficient to allocate it. Just think if all the memory referenced in the reserved column needed to be swapped out to your device - this would leave no device swap and the total count would be the same. It needs to gaurantee there is sufficient device swap there before allocating it.

Cheers,

James.

[James Murtagh]

Oops, maybe never read the question exactly right. Processes need to reseve (virtual) memory before they use it so at any time there may be memory reserved but not in use. Hence, there may be nothing to actually page out to the device. However, if it were then used by a process the kernel has already guaranteed the device swap is there to satisfy it in the event it is paged out. Adding more swap space will help add to the total vm available hence allowing more memory requests.

[Jeff Schussele]

Hi Sy,

The reserve area is like booking a table at a restaurant that you really don't want to go to but just *may* have to in case the wife burns the supper. You'd have to eat something & in this case the reservation is mandatory. So if you can't make the reservation, you can't eat at all.

Solution would be to add more swap - secondary is fine - you just need more & then *hope* you don't start paging out, because if you do the system is going to slow *way* down.

May be that you also should consider a memory upgrade or kernel parameter tuning to free memory for the apps.

Rgds,
Jeff

[DIPAK KUMAR ROY]

Pseudo swap is portion of physical memory used as swap. It can go max upto 75% which means if you have 4GB of RAM you can have max 3GB of pseudo swap if it is free.

Adding some dev swap will definitely help in your case.

Thanks

[Simon Hargrave]

So, using a restaurant analogy. On my server I have 70 seats (40 in smoking, 30 non-smoking ;) There's 16 people in smoking, 27 in non-smoking, and a further 24 booked.

Those extra seats may have been booked by large parties (processes), but only some of them have turned up. Some may arrive later (process actually starts using the memory it's malloced), others may be forever laying dormant.

So I take it you'll rarely ever be using all swap before exhausting memory, as processes reserve large amounts of memory they think they may need at the start, but don't necessarily use it all?

Have I got this right?

Clearly the solution is to add more seats - preferably non-smoking, but smoking will do ;)

[Jeff Schussele]

Yes, you have it down fairly well.
You have to have enough seats for ALL parties that will and/or *may* show up.
Can't give the seat to anybody else until they have consumed their meal i.e. the process has ended - even if they never showed up.

Options include:
1) Add more secondary swap - easiest
2) Decrease RAM demand of the apps - harder

I also suggested that you need to increase free RAM because your swapinfo IS showing usage - 40% in this case. You could reduce the need by reducing the RAM demand OR increase free RAM. Have you checked kernel parms (dbc_max_pct, etc) for wasted resources?

Rgds,
Jeff