topic Re: swap utilization is high in Operating System - HP-UX

swap utilization is high

Hanry Zhou — Mon, 14 Jun 2004 09:22:01 GMT

N class server with 6gb memory, and following is the output of swapinfo -tam:
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1024 0 1024 0% 0 - 1 /dev/vg00/lvol2
reserve - 613 -613
memory 4448 2905 1543 65%
total 5472 3518 1954 64% - 0 -
total 5472 3518 1954 64% - 0 -

The total of swap util seems to be 64%(so in glance), and 1024mb swap device is configured.

Here is the problem:
sometime, the system response time is slow, and sometime the ITO warning messages saying SWAP utilization reaches 90% high.

My questions are:
1. is there definition difference on swap util. between ITO and swapinfo command.

2. the swap space is configured too small? If it is too small, how come only 0% of device swap used since the system last time boot up?
The is the part troubling me most.

3. currently the total swap space is about 6gb (5gb pesuo memory + 1g device swap)which is 1:1 to physical memory. so, the swap spacw should be okay?

Thanks,

Re: swap utilization is high

Steven E. Protter — Mon, 14 Jun 2004 09:31:24 GMT

If swap utilization is hitting 90% you are probably either overloading the system or have inadequate swap space. You may wish to measure swap usage over tiem(utility attached).

1) there are small calculation differences between the various utilities for showing swap.

2) Swap must be a minimum of half of RAM. It is generally set to between 1.5 and 2.0 times RAM. If analysis shows you need more than this, your system is being overloaded and adding swap will simply slow it down.

3) See my answer to item 2.

Summary: Swap is too small, based on my experience.

SEP

Re: swap utilization is high

Hanry Zhou — Mon, 14 Jun 2004 09:49:29 GMT

How come the usage of device swap is 0% if I don't have enough swap sapce configured? I would expect the system would also swap processes over to the device swap, if the space of pseudo memory is not large enough for swapping?

Re: swap utilization is high

Ravi_8 — Mon, 14 Jun 2004 09:57:53 GMT

Hi,

the swap seems to be very less(6 GB physical and 1GB swap) atleast make swap half of physical memory.

Re: swap utilization is high

Hanry Zhou — Mon, 14 Jun 2004 10:01:45 GMT

Ravi,

We have also more than 5gb pseudo memory swap space, the the total is equal to the 6g memory. Isn't?

Re: swap utilization is high

Hanry Zhou — Mon, 14 Jun 2004 11:51:56 GMT

When we say that the swap space should be 1.5, 2, or whatever times of the size of Memory, does the swap space include Pseudo memory swap space?

In my case, I have >5gb psedo memory swap space, and 1gb device swap.

Re: swap utilization is high

Patrick Wallek — Mon, 14 Jun 2004 11:56:05 GMT

According to your swapinfo output, you are not paging out yet. I think that ITO is reading the "memory" line and getting confused.

As long as you do not actually see the "dev" percentage go above 0, you are OK.

Re: swap utilization is high

Hanry Zhou — Mon, 14 Jun 2004 11:59:53 GMT

Patrick,

YOu seems right, but how come the system response time is slow? there are not too many cpu activities, and i/o is not too intenisive either.

Can anybody please tell me if pseudo memory swap is part of swap space? and should be counted as part of 2 times of physical memory
?

Re: swap utilization is high

Ted Buis — Wed, 16 Jun 2004 18:11:17 GMT

No question in my mind you need to increase device swap to at least equal memory and more if you are getting to 90% as you indicated. Swapmem_on=1 is only good if, as in your present case, you have more RAM than device swap. Good discussion in the book, "HP-UX 11i Internals" by Chris Cooper and Chris Moore. You may be experiencing inconsistent response because of pageouts, but you have 5GB that is locked in RAM due to being used a pseudo-swap which really means that it is bound to RAM. That's fast for those application that happen to be in that RAM area (pure luck), but means that the remained of your applications in areas that can be paged out will get hit repeatedly when real memory is used up. Disk space is cheap, always have as much device swap as RAM.

Re: swap utilization is high

Sridhar Bhaskarla — Wed, 16 Jun 2004 20:46:27 GMT

Hi Hanry,

There are couple of things I would check if the system is responding slow. First the buffer cache. If it is set to 50% which is default and if it is all used, then the system might experience intermittent freezes. If it is at 50%, reduce it to around 5-10%. The other thing is to look for the network configuration. If it is on a 100MB network, make sure your lan card as well the network switch port are set to 100 Full Duplex - Auto Off.

May be there is some application process or something that is running intermettingly causing the swap to go upto 90%.

About your swap calculation, though you are running ok at this time, it is not a good idea to configure the device swap less than the size of memory. This is because, as the memory keeps getting locked, the used (or not available for reservation) column in the memory line will increase. At some point of time that may make the system to depend more and more on the device swap for reservation.

Also ensure that kernel parameters like nfile, nproc, ninode etc., are not configured very high than required.

-Sri

Re: swap utilization is high

Hanry Zhou — Wed, 16 Jun 2004 21:42:40 GMT

Sri,

1. Buffer cache settings is okay: 460mb among 6gm RAM.

2. It(system slow) happens very often, so it is not itermittent. 90% is based on ITO, but 64% total based on swapinfo -tam

3. network is fine.

4. parameter is okay, not error/complain about these kernel parameters.

MY questions is since device swap is used 0%, Do I really have to increase the device swap space, and why?

Re: swap utilization is high

malay boy — Wed, 16 Jun 2004 22:55:46 GMT

Harry,
If you swap space utlization is 0 and I don't think you need to increase it.You might be worried if you swap utilisation is 90% above.

my 2 cents.

regargs
mB

Re: swap utilization is high

Ted Buis — Thu, 17 Jun 2004 09:28:12 GMT

The reason you need device swap is the way that HP-UX does paging. It reserves page-out space at the time the application is launched. If sufficient space cannot be reserved, then the application will not run. Psuedo-swap basically, locks pages in memory so that they will not be swapped and so don't need to reserve device swap. This is done so that applications don't start and then die in the middle of something when they have to pageout but there is then no place to put them. The idea is that it is better for them to not run at all if there is a chance they will die from lack of this resource. The book on "HP-UX 11i Internals" by Chris Cooper covers this nicely in Chapter 7.

Re: swap utilization is high

Hanry Zhou — Thu, 17 Jun 2004 10:19:38 GMT

Ted,

thanks for your messages.

I believe what you are saying really got the point.

However, based on the result of swapinfo -tam, the device swap is just 0% used, so, can I conclude that I don't have the swap space pressure? I understand that if the process can not reserve the certain amount of swap space, it will not be launched, and an error message will come out. but in my case, since i don't see any error messages when the system is slow, so I can conclude the problem on this system is now swap space related.

Please advice.

Re: swap utilization is high

Sridhar Bhaskarla — Thu, 17 Jun 2004 10:28:05 GMT

Hanry,

There are two things to swap utilization. Utilization due to reservation and utilization due to paging. Second one is what you don't want and that's not happening on your system. But that doesn't mean you don't need swap space. Pseudoswap covers you for 75% of the swap. So, one would think that you need only 25% of the device swap. However, as the system is used, more and more pages may get locked into memory which will decrease the amount of memory available for reservation. So, at one point of time, depending on the memory usage, you may end up with insufficent swap for reservation as your device swap (which is static) that insures pseduoswap is low.

But that may not be a problem in your case.

I would ask as very basic question. When you said there were intermittent slow downs, are they related to the response to the commands or they really slowing down the system so that processing is delayed?

-Sri

Re: swap utilization is high

Ted Buis — Thu, 17 Jun 2004 11:26:43 GMT

Sridhar answered it, there is a difference between allocated, reserved and free swap.
Allocated swap is "in use", and reserved has be specifically set aside for running program pages that have not yet been paged out. If you have no free swap, you can run any more programs, as they cannot reserve any space to which to page-out. So it is possible to have none or very little free swap, but to have never actually have used (allocated) any swap at all. Reserved swap hasn't been "used".

Re: swap utilization is high

Ted Buis — Thu, 17 Jun 2004 11:30:36 GMT

Correction:
If you have no free swap, you can't run any more programs, as they cannot reserve any space to which to page-out.
Sorry, for the typo.

Re: swap utilization is high

Bill Hassell — Thu, 17 Jun 2004 12:23:04 GMT

swapinfo reports what is happening at the moment. It is trivial to write a program that allocates 1000megs of RAM, and run this program 6 times to use up all memory and swap, then kill the proceses. swapinfo will not 'remember' any of that. The key to swap is whether it is really being used. ITO just looks at swap allocation once in a while and may have seen a spike. You need to monitor whether any page outs atre ocurring (vmstat). 0-9=ignore, 10-30=monitor, 40-90=beg for more memory, over 90=demand more memory.

Note that you may not actually run any programs are swapped out, instead, massive use of mempry mapped files may be causing the swap usage.

Re: swap utilization is high

Hanry Zhou — Thu, 17 Jun 2004 15:10:28 GMT

Sri, Ted, and Bill, very well put.

Sir,
What is the difference between slowing to the response to the commands and slowing so that processing is dealyed?

In my case, I'm sure it was slow to my commands, like bdf..etc.
I'm not so sure about if the process was delayed.

Also, it was not intermittent, it last for quite long time.

So, even though my device swap is 0%, there is still the possibility of lacking of device swap space, therefore, I should add more device swap?

Re: swap utilization is high

Ted Buis — Thu, 17 Jun 2004 15:46:18 GMT

If reserved swap gets over 90% (allocated may still be zero), you have less than 100MB of free swap (new virtual memory). If you have dynamic buffer cache, I think (hope) the system is going to want to try to shrink that to free RAM, burning some cycles in extra overhead. Anytime virtual memory is short, system overhead will increase. Along these lines, when a file system gets close to full, performance also drops.