topic Re: High swap space usuage in Operating System - HP-UX

High swap space usuage

Bruno Bossier_1 — Thu, 21 Oct 2004 02:44:35 GMT

I have a 2-node Serviceguard cluster (11.16) with extensions for RAC (latest version) and running an Oracle 9i RAC (cluster interconnect via Hyperfabric using UDP).

Swap space usuage is constantly very high, over 90%. I have been looking for specific processes who might cause this, but could not find any, even after stopping Oracle RAC, Serviceguard, ... swap space usuage stays very high. After we reboot both machines, all is OK, but after while usuage increases again to over 90%.

Any ideas ?

Thanks,
Bruno

Re: High swap space usuage

Bharat Katkar — Thu, 21 Oct 2004 02:57:31 GMT

Bruno,

Could u go thr' this cookbook attached!!
Hope that helps.

Regards,

Re: High swap space usuage

Fred Ruffet — Thu, 21 Oct 2004 03:12:11 GMT

Are you using raw devices or FS to store Datafiles ?

Regards,

Fred

Re: High swap space usuage

Bruno Bossier_1 — Thu, 21 Oct 2004 03:25:12 GMT

We are using raw devices. All shared storage is configured using Veritas Cluster Volume Manager.

As for the cookbook, I will look into it, but it's gonna be heavy reading :-)

Bruno

Re: High swap space usuage

Fred Ruffet — Thu, 21 Oct 2004 03:49:06 GMT

can't you see processes using a huge amount of memory ? using
UNIX95=true ps -A -o vsz,pid,args
and
ipcs -mb

have you possibility to start with just one node to see if it acts the same ?

Regards,

Fred

Re: High swap space usuage

Steve Steel — Thu, 21 Oct 2004 04:33:11 GMT

Hi

The unix95 shown is a good start

Here is a script of 39 lines

1 #RSZ = Resident set size
2 #VSZ = Virtual set size
3 #
4 #RSZ is a subset of VSZ
5 #-the portion of the processes memory which is 'resident'
6 #in physical memory (ie not paged out).
7 #
8 #VSZ is the total size, including resident and non-resident pages.
9 #
10 #These values are totals, obtained by adding up the memory utilisation of the
11 #processes various memory segments (of which there can be lots).
12 #################################################################################
13 uname -a
14 echo -------------------------------------
15 /bin/rm /tmp/$PPID 2>/dev/null
16 if [ "$LOGNAME" = "root" ]
17 then
18 echo memory from kernel
19 name=$(file /stand/vmunix|grep 64)
20 if [ "$name" = "" ]
21 then
22 echo phys_mem_pages/D | adb /stand/vmunix /dev/kmem 2>/dev/null
23 else
24 echo phys_mem_pages/D | adb64 /stand/vmunix /dev/kmem 2>&1|grep -i page
25 fi
26 fi
27
28 echo " "
29 echo swap
30 echo -----------------
31 swapinfo -tm
32 echo " "
33 echo memory usage per process
34 echo "----------------------------------------------"
35 UNIX95= ps -e -o ruser,vsz,sz,pid,args > /tmp/$PPID
36 head -n 1 /tmp/$PPID
37 tail -n +2 /tmp/$PPID|
38 sort -rnk2
39 /bin/rm /tmp/$PPID 2>/dev/null

Also look at

Memory Usage - â What is using all of the memory?â

by:eric.herberholz@hp.com

Last modified: October 18, 2004

Full document is available at external ftp site: ftp://eh:spear9@hprc.external.hp.com/memory.htm
The "Table of Contents" is available in ITRC doc id MEMORYKBAN00000975

You can reach the ftp site shown and there is good info and tools available

Steve

Re: High swap space usuage

Bruno Bossier_1 — Thu, 21 Oct 2004 07:14:51 GMT

OK, thanks to all. We have also a call open with HP support, but will in parallel investigate using this script.

Bruno

Re: High swap space usuage

Bill Hassell — Thu, 21 Oct 2004 08:19:13 GMT

Swap usage means low memory or very high memory mapped file usage. Now the ps listing (UNIX95= ps -e -o ...) should total up the current size of process usage, but also check on shared memory usage:

ipcs -bmop

You may find huge shared memory segments, which is OK and really improves performance unless you have critically small RAM. If swap thrashing is occcuring (processes competing for memory) then vmstat's metric po (page out) will indicate this. Single digits OK, double digits so-so, triple digits for long periods = very bad performance and lack of RAM.