memory problem

[Luc Monnens]

Dear all,

I posted this yesterday under the 9000 server forum, but I guess this is more where it belongs.

I have an L3000 running hp-ux 11.0.
It has 4Gb RAM and 4Gb of Device swap.

Lately my available memory has gone down from about 3400Mb to 50Mb when nothing is running at night.
I have no idea why an idle system needs all that memory.

Because of that there is no memory left to run production during day time so it has to swap out like crazy.

Any input on what this might be AND/OR on how to discover what is constantly keeping memory occupied will be much appreciated.

One more thing: I looked for memory hungry processes and found mib2agt. From the forums I know now this has a memory leak, so I stopped it until I can patch it.
Strangly enough that had no impact whatsoever.

Anybody?

Kind regards,


Luc Monnens

[Joseph Loo]

hi,

need to appreciate what applications or database running on that server. how will u know if it is idle when u have nothing to monitor it during the night.

running a cron with:

vmstat 5 300 > /tmp/memory

will help u check if the free memory really drops and how much it does during the night.

regards.

[Luc Monnens]

Hi Joseph,

Idle means there is NO database activity or application running at night.
I'll try the vmstat tip.
Thank you.

Luc

[Geoff Wild]

How many processes are running?

ps -ef |wc

What is dbc_max_pct set to?

kmtune -q dbc_max_pct

You want that to be between 350 and 550 MB - which is 9 to 14 pct of your ram.

For checking processes, attached is a script called processmem - out put like:

# processmem mib2agt


Memory claimed by mib2agt: 1920 Kbytes.

Rgds...Geoff

[Luc Monnens]

Hi all,

One more thing to add.
At night - when nothing is running - the server has about 500Mb memory free.
During the day that goes down to 20Mb.
This info is correct, not my previous statement about 50Mb free with no load.
So there is a difference between the system being idle and running the application.
To me this means that my database and application use about all the memory they can get their hands on - which is ok.
It is the 3,5Gb that is constantly taken by something else that concerns me.
And appart from the mib2agt I stopped yesterday there is no real memory hogger process showing up anymore. As a matter of fact all top memory users are application processes now - and that is what one should expect.

Hi Geoff,

To answer your questions:
About 760 processes running currently.
dbc_max_pct is set to 10.
Thank you for your script, but I am afraid it will only be useful if you already have a suspect.

Kind regards,


Luc Monnens

[RAC_1]

Run the following commands through cron every 5 minutes. This will give you information about which process are running and how much memory they are using.

UNIX95= ps -ef -o "vsz,ruser,pid,args"|sort -nk1 >> /tmp/memlog
The value of vsz is in KB

vmstat 2 10 >> /tmp/freemem.log

The free memory unit is page. One buffer page is 4096 kb

Once you check which processes are causing memory bottleneck, you can start working on it.

It may be the case that a process could have memory leak and that is causing the problem.

[Joseph Loo]

u mentioned something about database and application. what is the database and version and also what is the application running on the server?

please also reply on this:
# ipcs -mbop
to check the shared memory segment

# kmtune -q shmmax
to check the max. shared mem segment.

regards.

[Luc Monnens]

Hi RAC,

I used the UNIX95 one already. That is how I found mib2agt.

I am not sure what the vmstat one is telling.
Could you elaborate a bit?
Thank you.

Hi Joseph Loo,

Like I said they do not come into this, because it is the idle system status that is keeping all this memory occupied.

Can you tell me a bit more what these two commands should show me?
Thank you.

Kind regards,


Luc Monnens

[Joseph Loo]

hi Luc,

as mentioned "kmtune -q shmmax" will query the kernel to check the max. shared mem segment allocated.

for "ipcs -mbop", it will check the shared memory segment which gives an indication of which user has taken up the shared memory. look at SEGSZ column for the allocated shared memory for each process.

for vmstat, it checks the virtual memory statistics of the server.

it will be good if u reply us on the output.

regards.

[Luc Monnens]

Sorry, forgot to close.