Shared memory > Physical memory

[José Enrique González]

Hi:

My SAP appl server (HP_UX 11.23, SD IA64 partition) is creating a total of 20GB in shared memory segments... but my server has only 16GB of RAM! Of course, after a while runnnig my free memory goes below 200MB and swap high-water value showed by swapinfo increases at big steps.

Despite the misconfiguration of SAP(obvious to me, although not a SAP expert) I wonder why operating system allows and applications to create segments over physical memory space. Is there a way to control this? Is there a limit for this over-locking of memory? Thanks a lot in advance for your comments about it!

Jose E

[Bill Hassell]

Virtual memory has been a feature for decades in Unix, even PCs allow more memory to be used than is available. That's what the swap space is for -- to act as a temporary storage for data when it will not fit. The steps to deactivate a process, then page out sections of memory to make room for more is a great solution for an occasional over-allocation, but it comes at a great cost: performance is terrible. You can very easily limit the maximum amount of shared memory for a single process with the kernel parameter: SHMMAX

I am sure that the SAP and database admins wanted the best performance possible, thus the configuration of 20 Gb for shared memory. However, the admins destroyed all the performance advantages by exceeding the available memory. The fix is easy: order another 16Gb of RAM and performance will no longer be affected by excessive paging.

[A. Clay Stephenson]

This is normal UNIX behavior and illustrates the difference between two different things that are commonly referred to as the same thing: swapping and demand paging. In the early days of UNIX, entire processes were suspended and written to disk along with critical process state data (this is actual swapping); later versions (those since about 20 years ago) use demand paging where only portions of the process were sent to disk to make room for others. This is why you are not physical memory limited butrather virtual memory limited. As you have seen this over-commitment of physical memory comes at a price. Performance has to suffer by at least a factor of 100. You need to either buy more memory or more sensibly tune SAP and the associated databases.

[Hein van den Heuvel · Accepted solution]

Smoke and mirrors!
It's all virtual memory.

As long as no program touches pages in those shared memory sections it can all happily not be instantiated and only pretend to exist. Once it is touched something gotta give. Stuff will be paged out. At a slow rate that may be sustainable, at a heavy rate it will bring the system to its knees.

In SAP terms you should probably simply look at ST02 and the 'maximum extended memory available/used' that's probably where the bulk of the allocation goes and that's where you can check whether you needed it all.
Be sure to check over all active instances.

If you do need more memory than available then it is probably better to adjust the SAP memory, and perhaps database buffer memory, downwards and let SAP and the DB deal with less. They are probably better set up to so so with their own PAGE/ROLL/TEMP techniques than the OS can cope.

hth,
Hein van den Heuvel
HvdH Performance Consulting.