memory leak

[Jerry_32]

We have a L3000 server for software development, this server has 4G RAM and 12G swap running hpux11, recently it ran out of swap space every two weeks, we suspect there is some program has memory leak, with ps -efl and top we couldn't find any unusual process, is there any other way to find out who cause the leak? Thanks.

[Hai Nguyen_1]

Jerry,

Can you run:

# top -d 1 -f /tmp/top.txt
# swapinfo -tam > /tmp/swapinfo.txt

And post them here.

Hai

[Chris Wilshaw]

You could monitor for processes in top with increasing values of SIZE and RES, or you could use

UNIX95= ps -e -opid,vsz,args

this will give you the PID, size in KB of the process and the command, with arguments.

It's also possible that the process size could have exceeded the value of maxdsiz kernel parameter.

[Hai Nguyen_1]

Jerry,

Can you post the output of sysdef as well

# sysdef > /tmp/sysdef.txt

Hai

[Jerry_32]

Hai,
here is information from: top/swapinfo/sysdef all put into one file.

[Hai Nguyen_1]

Jerry,

Found something. Your swapinfo output says that out of 12GB swap space, about 9GB (9266Mb) has been reserved for active processes. That is quite a lot.

Can you rerun the top command for me as follows:

# rm /tmp/top.txt
# ps -ef | wc -l
# top -d 1 -n -f /tmp/top.txt

and:
# kmtune -l > /tmp/kmtune.txt

and post them here.
Hai

[Jerry_32]

Yes,
I guess someone used the share memory never released, but how to find who use the share memory? here is the info attached.

[Hai Nguyen_1]

Jerry,

Can you post this output as well:
# ipcs -mbop > /tmp/smem.txt

Thanks.

Hai

[Jerry_32]

here you are.

[Hai Nguyen_1]

Jerry,

Here is what I see.

- your ipcs output is normal.
The total shared mem required by big users oracle, rgsoln* added up to roughly 1GB out of 4GB RAM which is, I believe, acceptable.

From the top output, I see:
- Many users spawns quite a few xterm, tcsh sessions which each requires roughly 1M swap space. Adding them up may result in a big sum of needed swap space. However, they are still not big swap-consumers here.

- There are several DBs of which each process consumes about 30MB swap space. Again, they are not an issue here.

- Now come the main eaters of swap space, rgsoln* users. I counted roughly 61 rgsoln* processes from the top output, each of which consumes about 110MB swap space. Do the math, you will see that together they occupy more than 6GB out of the 9GB swap space currently allocated to processes. I believe that the processes themselves play the main role in your swap space issue.


Now come some recommendations:
- Reduce the size of rgsoln* processes if possible. Or:
- Create more swap space.

Hai

[Jerry_32]

Hai,
Thanks for your help.
I will keep watching these process.
Jerry