topic Re: memory leak in Operating System - HP-UX

memory leak

Jerry_32 — Fri, 26 Jul 2002 12:15:33 GMT

We have a L3000 server for software development, this server has 4G RAM and 12G swap running hpux11, recently it ran out of swap space every two weeks, we suspect there is some program has memory leak, with ps -efl and top we couldn't find any unusual process, is there any other way to find out who cause the leak? Thanks.

Re: memory leak

Hai Nguyen_1 — Fri, 26 Jul 2002 12:22:09 GMT

Jerry,

Can you run:

# top -d 1 -f /tmp/top.txt
# swapinfo -tam > /tmp/swapinfo.txt

And post them here.

Hai

Re: memory leak

Chris Wilshaw — Fri, 26 Jul 2002 12:23:40 GMT

You could monitor for processes in top with increasing values of SIZE and RES, or you could use

UNIX95= ps -e -opid,vsz,args

this will give you the PID, size in KB of the process and the command, with arguments.

It's also possible that the process size could have exceeded the value of maxdsiz kernel parameter.

Re: memory leak

Hai Nguyen_1 — Fri, 26 Jul 2002 12:23:58 GMT

Jerry,

Can you post the output of sysdef as well

# sysdef > /tmp/sysdef.txt

Hai

Re: memory leak

Jerry_32 — Fri, 26 Jul 2002 12:37:39 GMT

Hai,
here is information from: top/swapinfo/sysdef all put into one file.

Re: memory leak

Hai Nguyen_1 — Fri, 26 Jul 2002 12:57:29 GMT

Jerry,

Found something. Your swapinfo output says that out of 12GB swap space, about 9GB (9266Mb) has been reserved for active processes. That is quite a lot.

Can you rerun the top command for me as follows:

# rm /tmp/top.txt
# ps -ef | wc -l
# top -d 1 -n -f /tmp/top.txt

and:
# kmtune -l > /tmp/kmtune.txt

and post them here.
Hai

Re: memory leak

Jerry_32 — Fri, 26 Jul 2002 13:09:56 GMT

Yes,
I guess someone used the share memory never released, but how to find who use the share memory? here is the info attached.

Re: memory leak

Hai Nguyen_1 — Fri, 26 Jul 2002 13:17:52 GMT

Jerry,

Can you post this output as well:
# ipcs -mbop > /tmp/smem.txt

Thanks.

Hai

Re: memory leak

Jerry_32 — Fri, 26 Jul 2002 13:24:32 GMT

here you are.

Re: memory leak

Hai Nguyen_1 — Fri, 26 Jul 2002 14:13:24 GMT

Jerry,

Here is what I see.

- your ipcs output is normal.
The total shared mem required by big users oracle, rgsoln* added up to roughly 1GB out of 4GB RAM which is, I believe, acceptable.

From the top output, I see:
- Many users spawns quite a few xterm, tcsh sessions which each requires roughly 1M swap space. Adding them up may result in a big sum of needed swap space. However, they are still not big swap-consumers here.

- There are several DBs of which each process consumes about 30MB swap space. Again, they are not an issue here.

- Now come the main eaters of swap space, rgsoln* users. I counted roughly 61 rgsoln* processes from the top output, each of which consumes about 110MB swap space. Do the math, you will see that together they occupy more than 6GB out of the 9GB swap space currently allocated to processes. I believe that the processes themselves play the main role in your swap space issue.

Now come some recommendations:
- Reduce the size of rgsoln* processes if possible. Or:
- Create more swap space.

Hai

Re: memory leak

Jerry_32 — Fri, 26 Jul 2002 14:23:43 GMT

Hai,
Thanks for your help.
I will keep watching these process.
Jerry