swap space question

[Nyck_1]

When I go into sam and look at system properties its telling me that I only have 248MB of available swap, but when I run swapinfo -tam its telling me different:-

swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 16384 0 16384 0% 0 - 1 /dev/vg00/lvol2
reserve - 15920 -15920
memory 65506 17798 47708 27%
total 81890 33718 48172 41% - 0 -

According to the above I'm not even using any swap, what is going on here?

[Anka · Accepted solution]

o dev line(s):
+ are the actual physical swap device(s)
+ show if swapping has actually occurred. In other words, the PCT USED column in the dev lines represents the value last attained during a previous period of swapping.

This is analogous to the high-water mark that a flood leaves.
+ to check to see if swapping is currently occuring, use 'vmstat -v 5 5' to see if the 'po' (page outs) is sustained above 0.
o reserve line(s)
+ indicate how much of the swap device(s) has(have) been set aside for memory should it need to be swapped.
o memory line:
+ indicative of how much of pseudo-swap has been reserved
+ when present, indicates pseudo-swap is enabled (i.e. swapmem_on kernel paraemter is set to 1, which is the default.) The size of pseudoswap is calculated to be 75% of the size of RAM (a.k.a. memory.) In other words, it does not refer to acual physical memory use. Pseudo-swap was designed specifically for large memory systems for which actual swapping is never (or rarely) expected to occur, so thereâ s less need to use actual physical disk space for swap. For more information, see swapmem_on(5) , which reads.

o total line:
+ the PCT USED value shown in the total line indicates how much swap space has been actually reserved for swap. When this percentage gets near 100%, processes will not start up (unable to fork process) and new shared memory segments can not be created.

[Dennis Handly]

>According to the above I'm not even using any swap, what is going on here?

SAM is correct if it means "device" swap.
As Anka says, you have reserved all of that device swap but haven't written any of it.
You still have lots of pseudo-swap. And 48 Gb of unallocated total swap.

[Nyck_1]

This makes a lot more sense now, cheers as usual for the excellent responses!

This server currently has 64GB of ram & 16GB of swap, would you recommend I allocate some more swap or leave it to see how it goes?

[Venkatesh BL]

I think that would depend on the workload. If the swap space usage remain at < 90% during peak run, then, there is no need to allocate more swap.

[Nyck_1]

One of my users has come back with the following with regard to this swap question I had:-

The total field (virtual memory) includes physical memory and physical swap space. So even though the system has available virtual memory (total == physical memory + physical swap), it can't reserve any more space for new processes as all of the 16GB of physical swap space has been reserved already for existing processes.

Would you agree with this statement?

[Don Morris_1]

No, because the "memory" line in swapinfo's output is memory *swap* not physical memory. They aren't the same thing.

It is entirely possible to have tons of memory swap available and have no (well, hopefully little.. the system should try to avoid freemem 0 situations) free pages of RAM available (all it takes is enough device/FS swap to handle the virtual side and enough pages touched to require all the RAM).

Similarly, it is entirely possible to have tons of RAM available and be out of memory swap (though you should be out of swap as a whole) -- just eat enough virtual address space.

What is correct in that statement is that the system total virtual address space is determined partly by the total swap available on HP-UX. Exceptions are objects that use Lazy Swap reservation (rare except for Stacks on IPF), and virtual objects which are backed by existing files on a filesystem and whose changes should be visible in the file (since you can always "swap" to the file -- you don't need explicit swap there).

But as long as you have swap available, either device, FS or memory swap -- swap reservations will use it. Restricting the system to device swap would make memory swap completely pointless.

[Nyck_1]

This is what I currently have now:-

swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 16384 0 16384 0% 0 - 1 /dev/vg00/lvol2
reserve - 16260 -16260
memory 65506 22471 43035 34%
total 81890 38731 43159 47% - 0 -

So from the above output I'm assuming all is ok for the moment, would you agree?

[Don Morris_1]

As long as you don't need to create over 43Gb of virtual objects, sure. Certainly I wouldn't be worried at this point unless I knew I was about to try to bring up a 48Gb Oracle SGA or something.

[Nyck_1]

one of my users is running some perl program and it keeps on crashing with the following error:-

Unable to fork: Not enough space

and as you can see its not a swap space issue.

[Don Morris_1]

Presuming it isn't a fork bomb eating all the swap faster than you're monitoring it (and then going away), yes. Or that it doesn't malloc 48Gb or something nuts.

This is when knowing your OS version is nice to double check.

First question would be if there is anything special about this perl thing (besides is it a fork bomb, anyway) -- does it mlock memory? Is it a 64-bit process trying to fork a 32-bit child or vice-versa? Are Memory Windows in use in any way?

ENOMEM can also be returned for corner cases where virtual objects can not be created for other reasons such as protection id exhaustion (is there a lot of mprotect() activity going on as well?), but usually this is all about the swap space.

[Don Morris_1]

Oh, and are you using PRM, WLM or gWLM or any type of resource paritioning products?

[Nyck_1]

The machine in question is used pretty hard and there are 48+ oracle instances running at most times. Chances are they ran this perl script in the middle of the day and it went all wrong.

Someone mentioned the other day that they were unable to even run an ls.

They have managed to run this perl script on a another server but this one is a 2 cpu, 8GB of ram and not being used that much. The one is failing on has 8 cpu's and 64gb of ram.

[Nyck_1]

No resource partition products being used.

[Sreejith Kaliyam]

Hi,

Can you just double check the below parameters they are sufficient for your application requirement,since you have sufficient pages in psuedo swap for reserving and the systems is not even started paging .

maxdsiz
maxdsiz_64bit
maxssiz
maxssiz_64bit
maxtsiz
maxtsiz_64bit

Aneesh

[Dennis Handly]

>they were unable to even run an ls.

That would be a good time to get the swapinfo data but they may have to try several times.

>Aneesh: Can you just double check the below parameters they are sufficient for your application requirement

That wouldn't explain ls(1) above or the fact if it works some times.

>you have sufficient pages in pseudo swap

We don't know this is still true, when there is a problem.

>maxdsiz maxdsiz_64bit

Yes, if it isn't a fork bomb. But won't cause a fork issue, only malloc does that.

>maxssiz

This will reduce the heap area.

>maxtsiz maxtsiz_64bit

You should never have to worry about these.

[Anka]

The "cannot fork, not enough space" error is most often caused by a shortage of virtual memory or swap space.

When a process is spawned, the kernel will check virtual memory to see if there is space to accommodate the process. First the kernel checks that there is enough RAM available for the new process to run, that is not locked by other processes or used by the kernel. Then the kernel checks to make sure that the new process is able to reserve enough space in the swap area.

But as you have enough free swap because of the pseudo swap then i suppose it should not be the issue.

It will be good idea to check the free memory with "glance" before you run the perl script....or arround the time it terminates....
At the same time also #vmstat 1 10 ....pi/po

What HP-UX version do you have?

It also does not harm if you add an addtitional swap device - just for test purposes.

[Nyck_1]

All seems to be fixed now!

I had increased some kernel parameters earlier in the week and rebooted this server and there were also some other ones pending that required a reboot.

From what I understand is that the guy who had the issue might of been doing something wrong as no one else actually had this issue. So its one of those problems that gets fixed and your not 100% sure what actually fixed it.

The server in question is running 11.23.

Thanks for all the excellent replys!