PATROL detected 84.14/sec memory pages HIGH swap rate

[P Arumugavel]

Hi Experts,

Everyday I used to get alerts from BMC Patrol application which is configured for monitoring of my hp-ux boxes, that alert is "PATROL detected 84.14/sec memory pages HIGH swap rate".

OS Info: HP-UX B.11.31

#swapinfo -mnt
             Mb      Mb      Mb   PCT  START/      Mb
TYPE      AVAIL    USED    FREE  USED   LIMIT RESERVE  PRI  NAME
dev        4096     882    3214   22%       0       -    1  /dev/vg00/lvol2
dev        4096     312    3784    8%       0       -    1  /dev/vg00/swap
reserve       -    5797   -5797
memory    13545   11234    2311   83%
total     21737   18225    3512   84%       -       0    -

#vmstat 3 5
         procs           memory                   page                              faults       cpu
    r     b     w      avm    free   re   at    pi   po    fr   de    sr     in     sy    cs  us sy id
    3     2     0  1022992   13400    1    0    51    6     0    0     9   3975  39296  2160  20 13 66
    3     2     0  1022992   15131    0    0    64    3     0    0     0   2332 185752  1766  20  7 72
    4     1     0   970425   13543    0    0    32    1     0    0     0   2330 185483  1811  20  6 74
    4     1     0   970425   13319    0    0    16    0     0    0     0   2343 185272  1779  19  6 74
    4     1     0   970425   13401    0    0     7    4     0    0     0   2292 188093  2001  20  6 73

Please help me for permanent fix for this alert.

[Bill Hassell]

84 pages/second is a high number and it indicates that you do not have enough RAM. Double the amount of memory installed to 24 GB and add another 4 GB of swap space. The page out rate indicates that there is not enough memory to run everything at the same time, so less important programs are being deactivated and paged out to the swap area. You can also reduce the number of users or processes that are running and that will reduce the page out rate. If you are running a database, reduce the amount of shared memory configured for the database. That may stop the page outs but the database will run much slower.

Page outs (more than 0-20/sec) will be very detrimental to overall performance. Reducing memory usage can also slow down your system. There really isn't much choice except to increase RAM.

[P Arumugavel]

Hi Bill,

Thanks for the reply.

#vmstat 2 5
         procs           memory                   page                              faults       cpu
    r     b     w      avm    free   re   at    pi   po    fr   de    sr     in     sy    cs  us sy id
    2     0     0   975067  177856    1    0    51    6     0    0     9   3974  39405  2160  20 13 66
    2     0     0   975067  177144    0    0     1    0     0    0    30   2468 170514   409  18  7 74
    2     0     0   975067  176748    0    0     0    0     0    0    19   2473 171243   400  18  6 76
    2     0     0   948800  176388    0    0     0    0     0    0    12   2378 171065   412  17  5 78
    2     0     0   948800  175844    0    0     0    0     0    0     7   2439 169580   394  14  4 82

#  

Here, page-in and page-out rate is low.

In normal behaviour of system, how page-in and page-out rates should be?

Actually what do both indicate?

Rgds...

[Steven E. Protter]

Shalom,

Your system is 84% or 87% subscribed for memory according to the swapinfo snapshot.

 This is a little high and exceeds physical memory. So some processes are being swapped in and out of memory as they are needed by the CPU. This slows down your system, hence the alert.

Ways to deal with it:

1) Add RAM

2) Decrease demand for RAM. Oracle SGA requirements can be reduced. You can reduce the number of daemons running, you can decrease the size of the buffer pool. Any of these things will help.

root@dxd13nfs# kctune | grep cache
...

filecache_max                 8153993216  Default      Auto
filecache_min                  815398912  Default      Auto

The system above is not properly tuned. Yours may not be tuned either. Setting the filecache_max to a lower figure will provide you an immediate and possibly large positive impact on memory use.

SEP

[James R. Ferguson]

Hi:

---

@Steven E. Protter wrote:

Setting the filecache_max to a lower figure will provide you an immediate and possibly large positive impact on memory use.

---

This is not necessarily true.  The manpages for 'filecache_(max|min)' note that "It is recommended that these tunables are left in the automatic (default) state, to allow the system to better balance the memory usage among filesystem I/O-intensive processes and other types of processes."

Your mileage may vary.  An empirical evidence-based decision based on performance in your environment is necessary if you want to achieve the best results.

You might find the following whitepaper useful:

http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c02465072/c02465072.pdf

Regards!

...JRF...