Device Swap usage question.

[Gulam Mohiuddin]

Right now the memory (RAM) on our server is being 98% utilised.

Glance and GPM both shows Swap Device “/dev/vg00/lvol2” used 30MB.
But vmstat doesn’t show any swap/paging out “PO” activities.
Also swapinfo –tma command show device swap is 30MB used but the “% used” column shows “0”.

My questions:

1) Is our system paging out or swapping out to disk?
2) Will server have performance issues? (Most probably yes)
3) Why vmstat is not showing any “po” activity.

Our environment:

HP-UX 11i Version 1, 4GB RAM, Oracle8i databases 64-Bit, RP7

[Sridhar Bhaskarla]

Hi,

If your 'swapinfo -dft' is showing some usage in the "Kb Used" column, then definitely there are some pages sitting on the swap device. However, it could be due to memory mapped pages.

If the RAM is at 98%, it is not necessarily a bad sign until you start seeing pageouts in vmstat. On a memory starving system, you should see the 'po' almost constantly in double digits along with the "Kb Used" column growing in swapinfo -dft.

I assume you already trimmed your dbc_max_pct else system will try to make use of the memory until the parameter reaches 50% or when the memory usage nears 100%.

-Sri

[Eric Antunes]

Can you show us the results of:

swapinfo -t

and

vmstat -dnS 5 5

[RAC_1]

I think Shridhar have answered all your questions. Swapinfo gives infomration about swap space ussage and not about swp outs/ins.

If vmstat is not reporting swap ins/outs and vhand process is not in play, you are not swapping. Every process that starts will reserve some memory, so that in case if it needs to swap out, it has space availabe.

If alomost all of the swap space is used, you may get swapping outs, if more processes are started. It is time to increase RAM on your system. You can postpone that with some more swap space, but it will come with a performance hit.

Anil

[Gulam Mohiuddin]

rpludb03:/#swapinfo -t
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4194304 13644 4180660 0% 0 - 1 /dev/vg00/lvol2
reserve - 2688752 -2688752
memory 3154168 1044736 2109432 33%
total 7348472 3747132 3601340 51% - 0 -

[Gulam Mohiuddin]

rpludb03:/#swapinfo -dft
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4194304 13644 4180660 0% 0 - 1 /dev/vg00/lvol2
total 4194304 13644 4180660 0% - 0 -

[Eric Antunes]

You have very little swapping (13644) of all swap space available (4194304), that gives you 0.3% swap PCT used. The system shows you 0% because he rounds it!

[Sridhar Bhaskarla]

Hi,

Post your 'vmstat 5 5' output. YOu have around 13.5 MB sitting on swap which is nothing. It could be due to memory mapped pages as well. Also your "kmtune -l -q dbc_max_pct".

-Sri

[Gulam Mohiuddin]

So, is it bad or good?

May be nothing to worry about now but could be potential performance problem in future....

Gulam.

[Gulam Mohiuddin]

rpludb03:/#vmstat 5 5
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
4 1 0 203530 46689 6 3 0 0 0 0 0 1695 5952 486 11 2 87
2 1 0 250995 46518 65 0 1 0 0 0 0 1429 49984 396 34 4 62
2 1 0 250995 45941 20 0 0 0 0 0 0 1255 33129 287 31 1 69
1 3 0 257835 46713 40 44 0 0 0 0 0 1503 21064 377 26 3 71
1 3 0 257835 46713 13 13 0 0 0 0 0 1665 13001 415 16 3 82

[Gulam Mohiuddin]

rpludb03:/#kmtune -lq dbc_max_pct
Parameter: dbc_max_pct
Current: 15
Planned: 15
Default: 50
Minimum: -
Module: -
Version: -
Dynamic: No
rpludb03:/#kmtune -lq dbc_min_pct
Parameter: dbc_min_pct
Current: 5
Planned: 5
Default: 5
Minimum: -
Module: -
Version: -
Dynamic: No

[Sridhar Bhaskarla]

kmtune -l -q dbc_max_pct?.

The freemem is around 160MB. If your dbc_max_pct is at 50%, then it explains everything.

-Sri

[Gulam Mohiuddin]

I would like to re-confirm that:

Right now the dbc_max_pct is 15% and not at 50% as you mentioned in the last reply.

Gulam.

[Eric Antunes]

Humm,

You have too many processes blocked for resources!

Besides Sri request and since you're using Oracle, can you post you r init.ora and shmmax kernel parameter?

[Gulam Mohiuddin]

rpludb03:/#kmtune -lq shmmax
Parameter: shmmax
Current: 1073741824
Planned: 1073741824
Default: 0X4000000
Minimum: -
Module: -
Version: -
Dynamic: Yes

For init.ora, we have several Oracle instances on this box, at lease six production databases.

Thanks,

Gulam.

[Sridhar Bhaskarla]

Hi Gulam,

I posted my last message almost at the same time as of you.

600 MB is set aside for buffer cache, which isn't bad. 11i's kernel does quite a bit of dynamic allocation (more for JFS inode cache) so you will see the 'system memory' much more than that of 11.0. How is the contribution of memory in glance?. I mean Sys Mem: User Mem: and Free Mem: in glance. You can also get the memory usage using /usr/contrib/Q4/bin/kmeminfo.

I believe the system is almost at the edge though the system is doing fine at this instance.

I would keep a close look on it (particularly the po column in vmstat). Particularly when there are six databases running, I would keep some memory in hand to add it in case needed.

-Sri

[Eric Antunes]

You are using the same value as me but I've only 1.5Gb of memmory and you have 4G, so I think you should triplicate this value to 3221225472!

[Gulam Mohiuddin]

Thanks for your response.

But I am really confused about swapping.

How I will determine that my server is paging out or swapping out, which command will provide this information to me.

Thanks,

Gulam.

[Eric Antunes]

Sorry, I meant duplicate to 2147483648. And check your nfile if it has something like 12000.

Regards,

Antunes

[Bill Hassell · Accepted solution]

To see if swapping (paging) is occurring, use vmstst and look at the column po )page out). Ignore page in (pi) as it counts programs starts as well as page in. You'll need to monitor this over time (vmstat 30 30) which will report every 30 seconds for the next 15 minutes. A non-zero po value means a page-out. Single digits (0-9) can be ignored. Double digits (more than 10 for long periods) are a concern that RAM is too small, triple digits means that a *lot* of swapping is taking place.

One other note: mamory mapped files may be in use on your system and this will consume some swap area. That is normal and not a performance issue.