topic Re: memory / swap problem in Operating System - HP-UX

memory / swap problem

so_2 — Wed, 25 Mar 2009 11:04:50 GMT

Hi Gurus,

I have rp7420 server running HPUX B 11:11.
The memory in this server is utilised 99% almost all the time.
But I can not see swap is utilised for this server.No page in page out found.

My queries are below.
1)Why eventhough there is a memory bottleneck, the swap space is not utilised?
2)When I run a tsm backup it fails as the system unable to allocate memory for tsm read. Why the swap case is not helping here?
3)How can I fix this?

Note : There are no performance issue with the server.

Please find the attachment for more details

Thanks
SO

Re: memory / swap problem

Roman Schmidt — Wed, 25 Mar 2009 11:55:29 GMT

hi

this is ok, the memory is used by caches and buffers.

if a process needs memory, this portions of memory will be freed automatically for the process needing it.

in linux you have following command:

$ free
total used free shared buffers cached
Mem: 2074792 440016 1634776 0 49852 306576
-/+ buffers/cache: 83588 1991204

here you can see the "real" memory utilization.

maybe there is such a command in hp-ux too.

Re: memory / swap problem

Roman Schmidt — Wed, 25 Mar 2009 12:04:01 GMT

use

# glance -m

there you have for example following entries:

Phys Mem: 12.0gb
Buf Cache: 7.2gb

you see that most of the space is used by buffers, this will be free if necessary, so you can count them as free as well.

Re: memory / swap problem

likid0 — Wed, 25 Mar 2009 12:17:08 GMT

From your swapinfo, you could be running out of reserve swap, so new processes that whant to start on the server can't reserve enough memory in swap to start.

If you are getting fork and out memory errors in syslog.log, then thats the case.

You can add more swap, to get you going for the moment,but when vhand starts swaping out to disk, the performance is horrible, but that server needs more ram on the long run.

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 0 4096 0% 0 - 1 /dev/vg00/lvol2
dev 4000 502 3498 13% 0 - 0 /dev/vg00/swap2
reserve - 5980 -5980
total 8096 6482 1614 80% - 0 -
#

Re: memory / swap problem

Ganesan R — Wed, 25 Mar 2009 12:33:30 GMT

Hi,

>>1)Why eventhough there is a memory bottleneck, the swap space is not utilised?<<

How did you say memory bottleneck?. It is just 99% used not 100%. Still you have 1% memory. No swap activity will happen as long as the number of free pages remains larger than a predefined minimum limit known as "lotsfree".

>>2)When I run a tsm backup it fails as the system unable to allocate memory for tsm read. Why the swap case is not helping here?
<<

What is the error you are getting when you start tsm backup?

Re: memory / swap problem

Md. Farhan A Azam — Wed, 25 Mar 2009 14:22:45 GMT

Hi,

From kmeminfo out put, toatl memory in your server is 8 GB, and 6.1Gb (76%) is being used by the users. So you
can check the running processes for high memory utilization. If possible plz post the output of "top'

Thnx...Farhan

Re: memory / swap problem

Dennis Handly — Wed, 25 Mar 2009 19:27:17 GMT

You should enable pseudoswap so it shows in swapinfo: swapmem_on

>Why even though there is a memory bottleneck, the swap space is not utilised?

Why do you say bottleneck, since you say no performance issue? You do have a limit.

>2) When I run a tsm backup it fails as the system unable to allocate memory for tsm read. >Why the swap case is not helping here?

Because you need more than the 1.6 Gb you have free?

Re: memory / swap problem

John Jimenez — Wed, 25 Mar 2009 20:02:17 GMT

What database are you using? I think they can tweak that down max out at less. We run Oracle and I had the same issue. Even when no one was on the system it was at 100%. Because of an Oracle paramater, the databse was told to put every thing in memory.
Off the top of my head, I forgot which Database paramater they tweaked out and lowered. But when they did now it. memory is still always constant, but now it is at a constant 85%, not 98% like before. When they lowered it the D.B. seems to run as well as before.

Re: memory / swap problem

so_2 — Thu, 26 Mar 2009 10:11:12 GMT

Thanks a lot to all for their wonderful response. I have more queries and a effort has made to put more info to you
wherehever requeted. Apologies if I am wrong or infor is not adeqaute anywhere.

The desfree ,memfree and minfree values are as below.
Theoriticaly the swap will not happen until entire memory is exhausted. Am i right?
What is the recommended value for these?

# kmtune | grep -i free
desfree - - 0
lotsfree - - 0
minfree - - 0
#

The memory details are already provided with kmeminfo output.buffer cashe is 363 MB which seems normal.
There are only 8Gb memory and 8Gb swap. Adding more memory and enabling psuedo swap seems help bring the
% utilisation of memory down.But is there any other method for me to make the swaping happen and avaoid
me adding more memory?

We are running sybase database in it. Sybase is consuming more memory.Had a check with the database team,
but they says everything is normal at their end.

#UNIX95= ps -eo vsz,ruser,pid,args | sort -rn | more
55360 sybase 3886 /opt/sybase/ASE-15_0/bin/dataserver -ONLINE:2,0,0xffffffffc7d02
55360 sybase 3817 /opt/sybase/ASE-15_0/bin/dataserver -ONLINE:1,0,0xffffffffc7d02
55360 sybase 2597 /opt/sybase/ASE-15_0/bin/dataserver -sATLASPRD -d/dev/prdavg02/

The tsmerror.log extracts
=========================

The backup happen for long time and finally fails with this error.Observed that /home/chklogs
has millions of files inside in diffrenet directories.Does that mean,tsm is trying to copy those
files in to memory but fails due to insufficinet memory?

03/25/09 13:19:59 ANS1520E Failure writing to the Tivoli Storage Manager error log: errno = 13, No child pr
ocesses
03/26/09 00:19:23 PrivIncrFileSpace: Received rc=102 from fioGetDirEntries: /home/chklogs /workplace/logs
03/26/09 00:19:23 ANS1999E Incremental processing of '/home/chklogs' stopped.

03/26/09 00:19:23 ANS1030E The operating system refused a TSM request for memory allocation.
03/26/09 00:19:23 ANS1512E Scheduled event 'P0400D' failed. Return code = 12.

Re: memory / swap problem

Avinash20 — Thu, 26 Mar 2009 14:12:49 GMT

Hi,

Let me check the logs you have posted..

Hmm,,.
What makes you think, swapping is not happening..
Yes, there is a memory pressure on the server

Physical memory = 2090240 8.0g 100%
Free memory = 19854 77.6m 1% <<---
User processes = 1591851 6.1g 76% details with -user
System = 474513 1.8g 23%

## We are having only 77 MB of free memory out of 8 GB

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 0 4096 0% 0 - 1 /dev/vg00/lvol2
dev 4000 502 3498 13% 0 - 0 /dev/vg00/swap2 <<===
reserve - 5980 -5980
total 8096 6482 1614 80% - 0 -

## Since /dev/vg00/swap2 is of PRI=0, this will taken as the first swap device and 13% of device swapping is taking place.

You could observe the output of

# vmstat 5 5

You will observe lot of page-outs(po)
This suggests that the pages are being swapped from memory to disk due to insufficient memory

Let me check you next question !!

Re: memory / swap problem

Avinash20 — Thu, 26 Mar 2009 14:15:16 GMT

We are running sybase database in it. Sybase is consuming more memory.Had a check with the database team,
but they says everything is normal at their end.

## There is no running away from the problem

Please post the output of

# ./kmeminfo -V
# ./kmeminfo -user
# vmstat 2 2
# swapinfo -tam
# ipcs -ma