11.23; easy question for pros: pseudoswap

[Adam Garsha]

Does reservation actually block/put-aside physical memory for pseudoswap. This question is hard to put in words, but I am trying to understand if physical memory will actually be "wasted" for pseudoswap space in an environment where my physical memory is larger than my swap space and I don't want to be paging ever.

Namely, say I've got 48GB's of physical memory and only 40GB's of swap space on disk. My concern is the whole pseudoswap business leads to some of my 48GB's be reserved for swap and hence unable to be used for new processes! I think that that cannot be true and that instead pseudoswap is a hack to provide a lazy-swap-like experience for processes.

Maybe more clearly, does reservation for paging ever block use of physical memory? Again, assume that I have plenty of swap space that is not allocated (but may be reserved). Here is a view at my current condition:

#garshaa@its-ps1:/home/garshaa/# sudo swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 40960 1643 39317 4% 0 - 1 /dev/vg00/lvol2
reserve - 36135 -36135
memory 49122 13374 35748 27%
total 90082 51152 38930 57% - 0 -

My concern is that I am "wasting" physical memory for swap space reservation and I am looking for warm fuzzy's that that is not true and that pseudoswap is just a trick to enable you to do lazy-swap like allocation.

[Steven E. Protter · Accepted solution]

Shalom,

You might find reading up will provide you some warmer fuzziness.

http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1285700667264+28353475&threadId=1162150

http://docs.hp.com/en/5965-4641/ch01s09.html

http://docs.hp.com/en/B2355-60130/swapmem_on.5.html

SEP

[Adam Garsha]

I'll swing back with points in a bit.

Thanks, I've been reading off/and/on most of the day about this subject. The problem is, every time I see a post that I like, e.g. A. Clay's in the first link, I see another post that contradicts it. Heck, I called support and asked them and the guy made things as clear as mud. Maybe I am dumb.

Perhaps, in simple terms what does that 13374MB of "used" memory in my original post's swapinfo mean? Does it mean that 13374MB of memory is begin used for paging? (that would not make sense... why pageout your pages to memory?)

Then I see posts like whitenights: 'If your total swap reservation exceeds available device swap and you do have pseudo swap enabled, then programs will not fail, but the kernel will start locking their pages into physical memory. If this happens, the number for "Used" memory swap shown in glance will go up quickly. We realize this is a real
head-spinner.'

I think this is what could be happening to me. I wish my disk drives were bigger, I'd just add dev swap and forget-about-it. But sadly it is a blade and I've got no room.

I'll read the other links now.

[Adam Garsha]

OK, maybe support redeemed themselves with this blurb:

"swap is enabled (i.e. swapmem_on kernel paraemter is set to 1, which is the default.) The size of pseudoswap is calculated to be 75% of the size of RAM (a.k.a. memory.) In other words, it does not refer to acual physical memory use!!!! Pseudo-swap was designed specifically for large memory systems for which actual swapping is never (or rarely) expected to occur, so thereâ s less need to use actual physical disk space for swap. For more information, see swapmem_on(5) , which reads:
In previous versions of HP-UX, system configuration required sufficient physical swap space for the maximum possible number of processes on the system. This is because HP-UX reserves swap space for a process when it is created, to ensure that a running process never needs to be killed due to insufficient swap. This was difficult, however, for systems needing gigabytes of swap space with gigabytes of physical memory, and those with workloads where the entire load would always be in core. This tunable was created to allow system swap space to be less than core memory. To accomplish this, a portion of physical memory is set aside as 'pseudo-swap' space. While actual swap space is still available, processes still reserve all the swap they will need at fork or execute time from the physical device or file system swap. Once this swap is completely used, new processes do not reserve swap, and each page which would have been swapped to the physical device or file system is instead locked in memory and counted as part of the pseudo-swap space."

I read this as: yeah, it is just like lazy swap (circa Tru64) until you start to reserve more paging space then you have on physical disk; afterwards, your pages get locked in memory for the process requesting the additional space and no additional reservations are made.

[Adam Garsha]

Stated even more clearly here:

Swap reservation policies with and without pseudo swap are the same as long as there is
real (device and filesystem) swap available. They differ when requests for more swap
space are made after all of the real swap space has been completely reserved. Without
pseudo swap, the further swap requests would fail preventing further process growth or
creation. With pseudo swap switched on, the system will allow further requests, but
instead of trying to reserve space on the real swap devices, the system will simply keep
track of how much memory has been reserved in a virtual kernel array, allowing a predetermined
limit of pseudo swap to be reserved before denying further reservations.
When the time comes to swap or page-out process pages to make space in RAM, swap
space will be allocated as usual from the real swap devices, and when that space is
exhausted the remaining pages in memory become unswappable, or locked, until more of
the real swap space has been freed-up due to process completion or termination.
Pseudo swap has usually been a confusing subject because it is often mistakenly
perceived to be the same as device swap or filesystem swap. It is not! Pseudo swap is
really only a fudge counter for swap *reservation* purposes. It gives the illusion that
there is more swap space than there really is and thus allows the system administrator to
configure less disk space for swap.

When pseudo swap is enabled roughly 75% of the amount of physical memory is added to
the swap counter for reservation. This is what gives the illusion that we have more swap
than is really available. Note this pseudo swap is for *reservation* purposes only! We
cannot allocate or use this pseudo swap by paging to it. It is meaningless to page memory
to memory. The 75% figure was chosen as a guess of how much space user processes
would be consuming in resident memory (on a fully loaded system) and therefore would
not need to be swapped/paged to disk. Obviously if the kernel uses up more memory or
processes lock more memory pages then this figure should be decreased. As processes
allocate memory, they need to make sure that there is enough swap (real or pseudo) by
checking the swap counter. If there is enough then the counter is decremented. Reserving
swap is basically just that - checking and decrementing the swap counter. There are
actually two swap counters to keep track of, real and pseudo swap: swapspc_cnt and
swapmem_cnt respectively. We always reserve from the device swap counter
(swapspc_cnt) before decrementing the pseudo swap counter (swapmem_cnt). If the
system experiences memory pressure then memory pages need to be swapped/paged to
disk to ensure that there is an adequate amount of free memory. When swap disk blocks
are actually written to, we typically refer to them as swap being 'USED'. 'USED' is the
heading with which swapinfo labels the column. However, the term 'USED' can be
confusing when dealing with pseudo swap; after all, there are no disk blocks involved in
pseudo swap! Remember pseudo swap is only used for *reservation* and never for 'disk allocation'.

[Adam Garsha]

So the 13374 "USED" memory, is actually memory that has been locked by the kernel or by user procs and is no longer available for reservation. Nothing more, nothing less. Phew!

[Dennis Handly]

>if physical memory will actually be "wasted" for pseudoswap space in an environment where my physical memory is larger than my swap space

No. Pseudoswap is designed to take advantage of this case.

>my 48GBs be reserved for swap and hence unable to be used for new processes!

It's only used for "swap" each page to itself.

>instead pseudoswap is a hack to provide a lazy-swap-like experience for processes.

Not really. lazy-swap-like says "stab me in the back when I run out of memory". Pseudo-swap says, "don't let more guys run, if they want more".

>Does it mean that 13374MB of memory is begin used for paging? (that would not make sense... why pageout your pages to memory?)

Not really. Right. They are being used as active text and data pages, that "swap" to themselves.

>I'd just add dev swap and forget-about-it.

You'll may sorry since the performance may go way down

[Hakki Aydin Ucar]

Pseudoswap is enabled by default with the kernel parameter swapmem_on(5) set to 1. Don't worry about this line, just look at the total line for total used and sometimes it's interesting to look at the device PCT USED as an indication of how much swapping has occurred since the box was last rebooted. The memory line is also described in the swapinfo(1m) man page.
o To read more on pseudoswap, refer to:
Pseudo-Swap Space section of the Memory Management white paper
"What is swap and why does a system need it?" ITRC doc id KBRC00001262
"SYSADM: Swap and Pseudoswap Clarified" ITRC Doc id KBRC00010136

There is an attachment at the end of the post, get it.
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1089175

[chris huys_4]

Hi,

> >I'd just add dev swap and forget-about-it.

Then why dont you do just that. ;)
Device Swap doesnt have to be part of vg00, afaik. ;)

For the rest, how much device swap the system needs, if its reserved or allocated, forget about all that, just ask the kmeminfo utility from hp support and check kmeminfo and kmeminfo -user output on the system in question and its immediately clear, how much total device swap all the userprocesses are "taking" on the system, and thats the amount of device swap, that the system imho should have defined on the system, kmeminfo will also show how much total physical memory all the userprocesses use, and where the other physical memory is used for, and plan accordingly.

Oh yes, the USED device swap of 4% in swapinfo, is probably due to memory pressure, thats where I would be worried about. ;)

Greetz,
Chris