Why Not Put All Your Swap Into LVOL2 / Primary Swap

[Michael Steele_2]

Where is the mistake in this when you're eliminating the round robin writting algorithm to 2ndary swap.

For example
vg00/lvol2
vg00/lvol10
vg01/lvol11

HP-UX does not write similtaneously to these three swap vols. From my understanding its written round robin:

Going up: lvol2 then lvol10 then lvol11
Going down: lvol11 then lvol10 then lvol2

http://docs.hp.com/en/B2355-60130/swchunk.5.html

This means that more swap is allocated to each device (or file system) using the round-robin interleaving scheme when all priorities are equal.

Like alot of comments please - points promised

[Rita C Workman]

hmmm....

You'd have to build (or rebuild) the O/S to extend lvol2. OK - doable

My boxes have the O/S on internal disk, I don't have my primary or alternate boot disk on any SAN (maybe an HA boot as third option). But I always put my additional swapdisk on SAN attached storage.

So vg00 is on internal disk, but if I hit a swap issue and all the swap were on that internal disk then the round-robin writes would increase the disk I/O by 3 on internal disk.
But...if I keep the swap extensions on the SAN attached storage, it doesn't hit that I/O crunch.

Of course...the object is to ensure you don't page out to start with, by having sufficient memory that is properly tuned.

OK...that's how I see it. But, that's just my 2.5 cents.

Kindest regards,
Rita

[Pete Randall]

> Why Not Put All Your Swap Into LVOL2 / Primary Swap

Indeed, why not? Unless I end up in a situation where I need to expand swap, I put everything in primary. Like Rita pointed out, the real goal is not to swap, and worrying about I/O delays when you are swapping is pointless. As Clay used to say, you're just rearranging deck chairs on the Titanic at that point.


Pete

[James R. Ferguson]

Hi Michael:

Since you don't want to actually do swap I/O in the first place by having adequate memory and by not deliberately overloading your server, the configuration of swap space for other than process reservation is somewhat moot to me. Once you start doing actual swap I/O (page-outs) you're suffering degraded performance.

There is no difference between primary and secondary device swap insofar as it's all "device swap". Primary swap must simply be present a boot time and its automatically activated.

That said, I prefer to configure any necessary secondary device swap with a numerically higher value for its priority than for the primary device swap. This means that secondary swap is used in preference to primary swap space.

I would never setup two device swap spaces on the same physical volume with equal priorities. This could cause performance degradation when swap interleaving caused disk head movement to oscillate across a disk.

Regards!

...JRF...

[Michael Steele_2]

OK Rita - you have 64 GB of RAM and make and lvol2 150% or 96 GB to allow for growth. You've check you process memory consumption and you'll consume 60 GB.

Why is this not faster and more efficient than having having one or two additional lvols?

Why wouldn't it be faster by writing sequentially to primary instead of round robin?

[Michael Steele_2]

James: "...There is no difference between primary and secondary device swap insofar as it's all "device swap". Primary swap must simply be present a boot time and its automatically activated...."

Isn't writting round robing slower as stated in the swchunks reference?

Aren't you going to make swap at the minimum the size of your RAM?

[Torsten.]

If I assume vg00/lvol2 and vg00/lvol10 are on the same disk, the result is likely different if you use vg00/lvol2 and vg01/lvol11 only.

[James R. Ferguson · Accepted solution]

Hi (again) Michael:

> Isn't writting round robing slower as stated in the swchunks reference?

I assume, yes, since I tend to trust the manpages. Again, though, is "slower" something that will be measurable to me or to the poor end-user if I'm actually swapping? The point is I don't want to swap.

> Aren't you going to make swap at the minimum the size of your RAM?

Yes, that's my choice when I cold-install. Given the vastly larger disk sizes today, compared to (even) ten years ago, having a comfortably sized vg00 with a "fat" primary device swap isn't too hard to achieve. If I add more physical memory later, I would just add a second device swap on another volume group with a different priority value. My preference is to keep I/O on vg00 to a minimum under all conditions. I'd probably steal some space from one of my SAN disks for the secondary device swap.

As with all things performance-related, YMMV. TMTOWTDI always applies, too.

Regards!

...JRF...

[Michael Steele_2]

Torsten: True you would have 2 vg00 spindles, typically, and 2 vg01 spindles.

Questions: You're still round robin, would the spindles be enough to be faster than sequential writing in the same config?

[Michael Steele_2]

Thank you All so far - Continue comments will be awarded

[Rita C Workman]

Hi Michael,

Well as Pete & Jim ( & Clay)so aptly say - it doesn't matter. Why move the deck chairs around when the ship is sinking?

So, yes, put it all on lvol2.

Somethings are just a matter of 'taste'. Which is why I just add some quick disk. As sometimes my internal disk isn't that big, so I don't want to take up more with swap when I might need to extend /opt or /var, etc.

And we all agree that it's the expertise in the tuning of the parms that makes us look good!

So maybe next new server I stick all the the swap on lvol2.
...........
nah....I am a creature of habit! teehee

Regards,
Rita

[Patrick Wallek]

>>Isn't writting round robing slower as stated
>>in the swchunks reference?

If you get to the point of actually paging out what does it matter? Performance will suck anyway.

>>Aren't you going to make swap at the minimum
>>the size of your RAM?

Not necessarily. Only a few of my servers have "device swap" equal to RAM size. All others have device swap equal to 25-35% of RAM and have "swapmem_on" turned on so that I can still use all of my RAM.

Having device swap equal to RAM size, especially in servers with 128 GB or more of RAM, is a colossal waste of disk space in my opinion. Sizing swap to 1.5 - 2 times the size of RAM, even more of a waste.


Now, the way I read the document, swap "performance" (if it can be called that) will be hindered if swchunk is increased to allow more swap to be allocated. This would also appear to only matter **IF** some pages had already been written at the original swchunk size.

Also, the man page says "...reduces the possibility of one single device becoming a bottleneck..." So interleaving is desirable, but I would probably NOT put all device swap LVs in the same VG.

However, if you get to the point where device swap is actually used, you just need to buy more RAM.

[Torsten.]

I agree, on our Dome with a bit more than 1TB RAM in a single OS instance we don't need / don't want to have 1 TB swap space ... ;-)

[Michael Steele_2]

Patrick:

"...All others have device swap equal to 25-35% of RAM and have "swapmem_on" turned on so that I can still use all of my RAM...."

Very interesting.

Example please.

[James R. Ferguson]

Hi (again):

> Torsten: on our Dome with a bit more than 1TB RAM in a single OS instance we don't need / don't want to have 1 TB swap space ... ;-)

Yes, I'm not used to playing with the big iron! As already noted, the whole concept of 'pseudoswap' enabled with 'swapmem_on=1' (which is enabled by default on 11.31), obviates the need to have very large swap device spaces. My comment of having a primary device swap space roughly equal to the memory size is based on the "smaller" footprints :-)

Regards!

...JRF...

[Torsten.]

I agree here too.

At least if you need a dump device.

But sometimes you may decide to not have a dump device large enough for the RAM.

Remember the SD can have up to 2 TB! Maybe we will see much more in future.

# /usr/contrib/bin/machinfo
CPU info:
64 Intel(R) Itanium 2 9000 series processors (1.6 GHz, 18 MB)
533 MT/s bus, CPU version C2
128 logical processors (2 per socket)

Memory: 981492 MB (958.5 GB)



I share the opinion, if you start to swap out, you don't really need to think about how make this slow operation a bit less slow.

[Patrick Wallek]

Here's an excerpt from an rp4440 running HP-UX 11.11. This server has 16 GB RAM, but I've got 6GB swap configured. No problems with the server utilizing RAM. According to glance I'm currently at about 50% RAM usage.

# swapinfo â tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 2048 0 2048 0% 0 â 1 /dev/vg00/lvol2
dev 4096 0 4096 0% 0 â 0 /dev/vg00/lvol9
reserve â 4691 â 4691
memory 12690 2505 10185 20%
total 18834 7196 11638 38% â 0 â

# grep â i phys /var/adm/syslog/syslog.log
Dec 6 10:47:38 hqudb03 vmunix: Physical: 16775168 Kbytes, lockable: 12968948 Kbytes, available: 14886436 Kbytes

[Patrick Wallek]

Damn Windows cut-and-paste! Sorry about the junk characters.

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 2048 0 2048 0% 0 - 1 /dev/vg00/lvol2
dev 4096 0 4096 0% 0 - 0 /dev/vg00/lvol9
reserve - 4691 -4691
memory 12690 2505 10185 20%
total 18834 7196 11638 38% - 0 -

# grep -i phys /var/adm/syslog/syslog.log
Physical: 16775168 Kbytes

[Michael Steele_2]

Patrick:

"...Here's an excerpt from an rp4440 running HP-UX 11.11. This server has 16 GB RAM, but I've got 6GB swap configured. No problems with the server utilizing RAM. According to glance I'm currently at about 50% RAM usage...."

How did you determine 6 GB? Is the formula repeatable / applicaible to all boxes? I.e., superdome with 1 TB of RAM.

[Patrick Wallek]

When the swapmem_on kernel parameter is turned on, it allocates approximately 75% of RAM to be used as "psuedo swap" for process swap allocation.

The memory line of 'swapinfo -tam' indicates the amount of pseudo-swap. In this case I have 12.4 GB (12,690 MB) of pseudo swap + 6GB of device swap for a total of 18.4 GB.

When this server was initially set up, I had 8GB of RAM with 2GB of device swap and swapmem_on enabled.

I upgraded to 16 GB RAM about a year ago. With pseudo-swap of 12.4 GB after the upgrade, and only 2 GB of device swap (total 14.4 GB) I needed more device swap if I want to be able to use all 16 GB of RAM.

The 4GB of additional device swap was a number pulled out of the air. I could have done just 2GB of additional device-swap and still have the ability to use all RAM.

[Michael Steele_2]

Patrick and all:

First of all I'm very happy that you were all online today - I wanted to hear from you all.

Patrick, especiall, and others:

I'm looking for a replacement swap formula similar to the 11.11 maxswapchunks formula. Substitutes suggest adding up all memory consumed by all processes running on the box. (* obviously this would have to be collected before the install and historicl *).

Would not the forula be Glance Memory report, and us total consumed to determine swap size?

[Michael Steele_2]

I'll start a new thread with this second question since an answer from the community is - yes, put it all on lvol2 if you can.

[VK2COT]

Hello,

Sorry for a belated contribution.
I was teaching Network Node Manager 8i
at a remote site last week and this week
I am teaching HP-UX 11.31 in the hospital
(they are migrating from OpenVMS to HP-UX).

Anyway, there is an old saying that
"a friend in need is a friend indeed".

If you put one large swap device on the
boot disk(s), here are the issues:

a) If you start swapping (for WHATEVER REASON),
your disk performance would be terrible.

b) Remeber, primary swap space is SHARED
with dump by default. So, if a large RAM
server is crashing, you will take certain
time to dump pages into dump/swap area.
And on the way to reboot, HP-UX is,
by default, designed to save crash into
/var/adm/crash (again, that is a default
place). So, you are WASTING significant
amount of boot time after the crash because
silly dump export needs to finish.

c) Due to contiguous block allocation,
primary swap cannot be easily extended.
Yes, I know all the tricks or
possible methods that include even DRD, but that is still an extra work.

So, why do the old things when this works
nicely, even in the dark times when
swap is needed:

a) Small swap device on boot disk(s), say
4-8 GB.

b) Create dump space(s) and ensure it/they
are not shared with primary swap!!!!!

c) Disable savecrash at boot time (hint,
check /etc/rc.config.savecrash file).

d) Add secondary swap devices on SAN.
Most certainly, do not add them on the same PVs. If possible, make them the same size
as primary swap (4-8 GB).

How many you need? I would not really jump
to formula 1x or 2xRAM. That is a silly old
method.

e) Monitor your performance and if you need more RAM, that is what you should deal with.

In the worst case, if the budget is tight,
simply add more secondary swaps on the SAN.

f) Ensure that secondary swaps have lower
priority that the primary swap. For example, all secondary swaps might have priority 0 and
primary swap might have priority 1...

That way, if you really start swapping, your
boot disk will never be abused.

g) If you really have a crash, you will
save close to 50% in reboot time.

Once you recover, you can run savecrash
manually (calmly and patiently).

Remember, we are not building or designing
servers for good times but also for unplanned
bad times. I try to build servers with idea
that they might work long after I am gone. So, plan carefully and regrets can be
avoided :)

Cheers,

VK2COT

[VK2COT]

I know the thread is closed, but I thought it
was worth saying what I had to say.

ITRC points are not the primary concern or goal
in my life...

VK2COT