More SWAP

[Larry Basford]

My systems are configured with 3GB mem and 1GB of swap.
swapmem_on kernel parameter is 1 (on)
I always see error messages that swap is not equal to ram. But I never see swap to disk. It will first deactivate processes but never swap to disk.

Why should I look for more swap space if it never gets used ?

The Oracle system has 15 GB Ram 1GB swap
What advantage would it have if I alocated 15GB of swap ?

[Mark Grant]

Urrrmm unfortunately, HPUX pre-allocates swap. That means that every time something starts running, it reserves space in swap even if it never uses it. This means that if you have no more swap left, you can't run any more applications, even if you have enough RAM. Oracle doesn't need pre-allocated swap for SGA and PGA which is probably why your system doesn't fall over every time you try and start a database but you can not be using all the RAM that you have.

Do the decent thing and increase your swap.

[doug mielke]

I think the swapmem_on parameter can disable the reserving of swap for processes. It's a good safeguard to have, but if you never swap, that disk space is just wasted.

[Mark Grant]

I didn't know that Doug, thanks. However, is it a good idea?

I know some other unixes e.g. Linux do not pre-allocate but they have other strategies for dealing with running out of virtual memory. What does a non-preallocating HPUX box do? Does it just stop, or worse, does some processes keel over and die corrupting stuff?

[A. Clay Stephenson]

In your case, with pseudoswap enabled you really have 12.25GB of process space (0.75RAM + SWAP); to fully utilize your 15GB (not counting kernel data structures) you would need 0.25RAM (3.75GB) of swapspace. I would increase to that level and if you are not swapping you are fine. Make sure that every swap space is mirrored so that your system can keep running with a failed disk.

Because many people use swap as dump space (I do not; they really have nothing to do with each other), you need to think about how much dump space you need. Typically about 25% of total memory is sufficient and it shouldn't be mirrored. I assure you that HP support does not want a 15GB dump and that it why the low 25% of address space is usually plenty.

[Don Morris_1]

Not exactly disabling it -- pseudo swap is just an accounting trick where the physical page of RAM is counted as the swap for that page... so these servers will be using their RAM -- minus some reservations the kernel thinks it may need but isn't actually using (pseudo-swap is limited to not be *all* the RAM to account for kernel usage).

[Don Morris_1]

Prior reply was directed at Doug's comment by the way.

Regarding the other question as to "what happens when pseudo-swap runs out".. there's no difference between that and running out of disk swap reservation. When swap reservation fails, the allocation causing the reservation gets failed. If the app can handle that, fine. If the app is being exec'ed, it will fail the exec. If the app was growing something that had Lazy Swap allocation set up, it usually gets killed (the price you pay for using Lazy swap allocation). Nothing should fall over and be corrupted.

[Mark Grant]

Don, so we still pre-allocate when using pseudo swap?

Sorry to hijack this thread but it is still related I guess.

[doug mielke]

My understanding of enableing pseudo swap was that it allowed the system to count ram as available swap. It seems then that the effect would be disableing the reservation of disk based swap space when a process starts, even if it is reserving ram.
If then, you don't reserve swap when a process starts, you are running the risk of running out of swap, which can sometimes be ugly. It's a risk.
On the other hand, if you've installed massive amounts of ram, (our users won't tolorate the performance hit if we ever swap) and are sure you will never swap, then why waste the disk space. We do have some systems with filesystem swap configured as some limited protection.

[Don Morris_1]

I think we're verging on a semantics debate, really. :)

The long and the short of it is that there is *no* difference in the reservation code until the very low levels based on the type of swap being reserved. The RAM being used for pseudo-swap is simply added into the total swap available figure and treated just like anything else. If we need to kick any pages out, the kernel can tell if the backing store is "real" or not and act accordingly. That's why I think it isn't precisely correct to say that reservation is disabled.

This is different from Lazy Swap (and doesn't run the risk of swap failure) because when the physical memory to handle any process allocation is reserved just like disk swap.. no one can steal it away. Effectively, you have physical RAM allocated to you -- you just don't know which pages they are yet (or have translations, etc.)

So there are worst case scenarios where you could have extremely large virtual allocations on a machine with little or no physical swap, which are representing very sparse physical usages (i.e. you mmap() MAP_ANONYMOUS for 4Gb and only use a page... and the whole box is like that)... which would result in a lot of RAM being underutilized (since it would be reserved for these beasts). Usage patterns such as this really should be using enough physical swap so that RAM goes to actual physical needs.
And now I'm rambling... sorry.

[Mark Grant]

Interesting ramble though Don :)

[Larry Basford]

OK
See if I get this right.
My 3GB system with 1GB swap and
swapmem_on kernel parameter turned on
I am using .75 x 3 = 2.25 + 1GB swap = 3.25
so I should be OK ?

Would it run better with swapmem off
and add 5GB of secondary swap ?

What do I watch to indicate I am into swap condition ?

Glance KB Paged Out ?
Deactivations ?


I apriciate all the answwers but am still confused.

[A. Clay Stephenson · Accepted solution]

It's not a question of whether you will "run better" but rather will you "run safer". By "safer" I mean that the box will run more processes (or allow existing processes to grow). Even with machines with small amounts of swap and large amounts of memory, the machine will simply refuse to fork more processes or memory allocation requests will be denied. A crash will not result. The question is also different for a box with 3GB of memory vs. one with 15GB of memory -- which is technically referred to "lots".

There is no question that allocating at least as much swap as memory is the safer choice and in those circumstances there is no need run pseudoswap at all --- its just a tiny bit of kernel overhead that is not needed. By the time you start seeing deactivations, you have lost the battle so the metric to watch is the pageout rate. Some pageouts will occur even without swapping if memory mapped files are in use.
The ideal pageout rate is 0; single-digits are fine; when rates get moderately above that its time to add more memory or reduce the load --- especially if this occurs for sustained periods of time.

The most important thing that you need to keep in mind that when the pageout rate becomes high, performance is going to be so bad that whether or not you can run still more processes is unimportant. You really need nothing more sophisticated than vmstat. I really consider the 'po' column the ONLY metric of interest in vmstat's output.

The whole purpose of pseudoswap (and remember, pseudoswap doesn't 'exist' anywhere; it's just kernel bookkeeping) is to allow processes to run without requiring that there be dedicated "real" swapspace. For machines with "lots" of memory this makes sense. Why buy a bunch of disks that will never be used -- especially when you should buy twice that much (mirroring)?

After going through all this, I still fall back on my method that has served me well.
Allocate a small amount of primary swap -- you must have some. By small (even with boxes with "lots" of memory), I mean something in the 512MB to 1GB range. Monitot your system (vmstat is fine) and add additional swap as needed. It's so easy to add additional swap (especially if you deliberately leave unallocated space in LVM) that it's really not worth worrying about. As long as you have 0.25MEM as swapspace and swapmem_on=1 set, you will be able to run just as many processes "well" (ie without paging out) as the person who allocates 1XRAM as swapspace.

There are, of course, instances where it does make sense to have much more swap than actual memory and tolerate swapping. For example, consider a system which might have hundreds (or thousands) of users who run essentially the same interactive program -- maybe a search engine or documentation server. The total number of active users at any one time is small although the total number of users might be very large. In this case, having enough memory so that pageouts never occur would be expensive and wasteful. Here the pageout rate would probably be in the single digits and so a bunch of swapspace makes sense.