Multiple swap files on the same device

[Ken Englander]

VK2COT

I understand your statement about sizing swap as a general view, but we must size it large at 1-2 times the size of memory because of our application and/or running an Oracle database where we typically load as much into memory as possible in the SGA.

How much swap do you have allocated for your large memory systems? Do you actually have dozens of 4 GB LUNs?

[Dennis Handly]

>Ken: but we must size it large at 1-2 times the size of memory because of our application and/or running an Oracle database where we typically load as much into memory as possible in the SGA.

If you want to load as much in memory as possible, you don't want to be in swap. It does no good to have that much swap because if you need it, it will run dog slow.

So what is the "swapinfo -tam" output of your system?

[VK2COT]

Hello,

Dennis already answered your 1 or 2 x RAM
for swap requirement.

If you really need that much swap, simply
add many 4 GB LUNs for swap devices.

Let's say, you have a server with 64 GB RAM.

Set primary swap to 4 GB on boot disks (make
sure swap is mirrored UNLESS BOOT DISK IS
SAN-BASED LUN, which might be RAID-1 or RAID-5 on SAN already).

If you want to use formula 2 x RAM for swap,
than add 31 LUNs of 4 GB for swap devices.

In total, you will have 128 GB swap and
yet, its performance will be perfect.
Plus, you will never cause disk I/O
load on boot disks if swap is used.
And you will not "waste" disk space on
boot disk for silly swap area.

And, if you ever add more RAM, you simply
add more 4 GB devices for swap to
match whatever rule you use. No need to
resize primary swap or set up additional
swaps which do not have same size as primary
swap...

Cheers,

VK2COT

[Ken Englander]

Dennis and VK2COT,

I understand you both to be saying it is not necessary to have as much swap as I am allocating. I'd like to know what it is that I do not understand. HP-UX is requiring this swap space so it can "allocate" enough swap to match our memory requirements even though it is not using it. Are we doing something wrong?

It is my understanding and confirmed by HP consultants who have helped us that we must have sufficient swap reservable.

VK2COT - I would hardly consider managing 31 LUNs in order to set up 128 GB of swap simple. That strikes me as a small nightmare. That is simply my opinion.

Thanks for your input!

[James R. Ferguson · Accepted solution]

Hi (again) Ken:

> It is my understanding and confirmed by HP consultants who have helped us that we must have sufficient swap reservable.

Yes, this is true insofar as sufficient swap must be available to be *reserved* when a process is first spawned whether or not that process ever actually uses swap space. As I originally noted, turning on pseduoswap 'swampmem_on=1' in releases prior to 11.31 (or simply running 11.31 where the kernel behaves intrinsicly as if it were on), means that about 75% of your physical memory is counted "as" swap space for the purposes of meeting swap reservation requirements.

You have indicated that you have pseudoswap enabled. Hence you don't need to waste great amounts of disk apportioning it to device swap. As Dennis noted, you don't ever want to actually swap. Pseudoswap was designed for servers with large amounts of memory. You should be able to use a smaller number of device swap gigabytes than you have physical memory and still run everything you need. As always, YMMV (Your-Mileage-May-Vary), so emperical measurements on your part would be best :-)

Regards!

...JRF...

[Ken Englander]

James and VK2COT - I believe the issue we have relates to lockable memory, that is required by Oracle for the SGA. According to HP, swapmem actually decreases the amount of available lockable memory, but adding more disk swap helps with it. So, we are stuck with having rather large swap areas.

[Dennis Handly]

>I believe the issue we have relates to lockable memory, that is required by Oracle for the SGA.

I wasn't aware you had to be lockable for the SGA? Is it optional?

>we are stuck with having rather large swap areas.

This seems to defeat the purpose of having large memory systems.

[VK2COT]

Hello Ken,

Yes, your Oracle DB requirements seem to be
counter-productive in regards to RAM and swap
sizing. I have seen some other weird
applications that also commanded high amount
of swap (even though they did not use it).

Nevertheless, the effort to set up multiple
swaps on SAN is small in comparison to
the benefits. After all, how often do we add
or remove device swaps? It is typically
done via simple Shell scripts at the time
when the server is built and not revisited
during lifetime of the server (if the design
is good).

Spend a bit more time on this and other
issues when the server is built and then enjoy life without panics and performance
issues later on :)

Make computers work on boring and repetitive
tasks (they never complain) :)

Of course, as other said, there is no
bullet-proof solution that work in all
environments. The magic in our jobs is
to adapt and listen!

Cheers,

VK2COT

[VK2COT]

Hello,

One important item I forgot to mention (that I teach in HP-UX courses).

There is a significant difference in number of devices that could be used for swaps in HP-UX. Take a look at this kernel parameter:

HP-UX 11.11, maximum value nswapdev is 25

HP-UX 11.23, maximum value nswapdev is 25

HP-UX 11.31, maximum value nswapdev is 1024

If, for example, you need to set up 128 GB
of swap on HP-UX 11.23, you could not use 4
GB for primary swap and each additional LUN
(you would need 32 device-based swaps). So, primary swap of, say, 8 GB would be more appropriate (and hence, 8 GB for additional
LUNs).

If you use HP-UX 11.31, having 32
device-based swaps is easy and fully supported.

So, the moral of the story is: 4 GB for the primary swap needs to be taken with a grain
of salt. It depends on which version of the
HP-UX O/S is used too...

Cheers,

VK2COT

[Ken Englander]

VK2COT - You make a valid point about only setting up swap one time, but setting up dozens of LUNs does not make much sense if we never end up swapping anyway. Also, I have to consider that my recommendations can affect literally hundreds of systems. The impact of such a recommendation can be major in terms of time required.

Dennis - I may have inadvertently left out the fact that our goal is to use async IO and I think that is what requires the lockable memory. Oracle will run without it.