topic Re: Leaking all over the place in Operating System - HP-UX

Leaking all over the place

Michael D'Aulerio — Wed, 31 Mar 2004 16:51:57 GMT

Hello all,
I have HP-UX 10.20 running on an HP744 and am having problems with memory leaks (I think). Our applications alloc and then free memory in chunks but the virtual memory usage on the system keeps increasing and the processes keep eating up more VM. Is there a know problem with HP-UX not releasing memory to the system even when the app frees it? Is there some kernel parameter that can be tweaked? Thanks for any help.

Mike

Re: Leaking all over the place

Michael Tully — Wed, 31 Mar 2004 16:56:54 GMT

Other than making sure that you have the latest patches installed for your version, there is not a lot you can do at the OS level. You will find that the application is the problem with the memory leak. The only way of fixing this is to schedule regular reboots, or get the application vendor to fix their problem. You could increase your shared memory (shmmax) but this I am sure will be a temporary solution only.

Re: Leaking all over the place

Bryan D. Quinn — Wed, 31 Mar 2004 16:58:09 GMT

Hello Mike,

I know there is a problem with the mib2agt eating up memory. Specifically with 10.20, I am not sure. I have had the mib2agt problem on 11.00. I would probably recommend updating your patch level first off, if they are not up to date....whatever that maybe for 10.20. Have you run the following command to see if you see anything eating a lot of memory:

UNIX95= ps -e -o ruser,vsz,pid,args | sort -rnk2 | more

Please note that you need to have a space between the = and the ps.

Hope this helps!

-Bryan

Re: Leaking all over the place

Nicolas Dumeige — Wed, 31 Mar 2004 17:03:05 GMT

The virtual pages marked as free by an executable can freed at the OS level later on. My point is virtual memory is a layer an abstraction maintend by the OS, so a lot of the memory can be allocated in the swap space and not on physical memory, making the memory freed less urgent. Does you box suffer memory shortage ?

Anyway, if you're facing a memory leak, the cause of the trouble is likely to be in the application code and not in the hp kernel, at least this is my bet.

Maybe you can use a debuger to run and profile the application.

Re: Leaking all over the place

A. Clay Stephenson — Wed, 31 Mar 2004 17:08:23 GMT

First, you need to understand that free() doesn't release memory back to the OS. It does put it back on the heap for reuse by the same process. Only if the malloc() request can't be granted by the freed heaped space is more memory allocated to the process. You can shrink a process using sbrk() but it is death to mix sbrks() and malloc(), calloc(), and realloc(). If you choose to use sbrk(), you must use only it and assume full responsibility for memory management. This is really no practical because so many library functions allocate memory using malloc().

If you have a reasonably well-patched version of 10.20 and libc is reasonably new (bearing in mind the last 10.20 Gen Release patchset was Dec. '01) then there are no big memory leaks. I'm betting on a leak in the application itself. If you are linking 3rd-party libraries (e.g. databases), you might look there.

Re: Leaking all over the place

Michael D'Aulerio — Wed, 31 Mar 2004 17:21:40 GMT

Thank you all for the education on how the OS handles memory management. We are not experiencing memory problems at the moment but it has plagued us in the past. At first we only had 256 MB on board memory and the system was constantly thrashing, swapping processes in and out from disk to memory. We solved that by doubling the RAM. Later we found that when a lot of apps were running we used up the Virtual Memory and new processes would not spawn. We doubled the Swap from 1 GB to 2 GB and resolved that. I'm concerned that some apps are eating up chunks of memory and that we may run into memory problems in the future. I will check that we have the latest recommended patch set but I guess the real problem is with the individual apps. Thanks for the help.

Re: Leaking all over the place

Brian Hackley — Thu, 01 Apr 2004 11:30:25 GMT

Mike,

Its helpful to find out where your virtual memory is going. Use the program "kmeminfo" to do this. If you don't have it, one way to get it is to install the 10.20 q4 patch PHCO_28068.

# ./kmeminfo.10.20
Warning: found 1 unmapped system pages

Physical memory usage (in pages):
physmem = 40960 Available physical memory
freemem = 13722 Free physical memory
Used = 27238 Physical memory Used:
System = 11523 by Kernel:
firstfree = 4107 for Text and Static Data
Bufpages = 4096 for Buffer Cache
pagepool = 0 available for buffers
Dynamic = 3319 for Dynamic Data:
MALLOC = 2723 MALLOC() buckets:
bucket[ 0] = 0 size 1
bucket[ 1] = 0 size 2
bucket[ 2] = 0 size 4
bucket[ 3] = 0 size 8
bucket[ 4] = 0 size 16
bucket[ 5] = 44 size 32
bucket[ 6] = 12 size 64
bucket[ 7] = 115 size 128
bucket[ 8] = 174 size 256
bucket[ 8] = 174 size 256
bucket[ 9] = 443 size 512
bucket[10] = 151 size 1024
bucket[11] = 409 size 2048
bucket[12] = 24 size 4096
bucket[13] = 38 size 8192
bucket[14] = 78 size 16384
bucket[15] = 117 size 32768
bucket[16] = 52 size 65536
bucket[17] = 32 size 131072
bucket[18] = 954 size 262144
bucket[19] = 80 size 524288
bucket[20] = 0 size 1048576
Kmalloc = 544 k[m]alloc()
Eqalloc = 44 eqalloc()
Reserved = 8 Reserved Pools
Other = 1 other system
User = 14679 by User processes
Reclaimable = 23 by Page Cache
Disowned = 21
Uarea = 492 by Ureas
Other = 741 not accounted...

Regards,
Brian Hackley

Re: Leaking all over the place

Jeff Schussele — Thu, 01 Apr 2004 11:46:00 GMT

Hi Mike,

As Brian points out, kmeminfo is a great utility - highly recommended.
I'd also recommend the equivalent utility for scoping out shared memory - shminfo - avail here:

ftp://contrib:9unsupp8@hprc.external.hp.com/sysadmin

HTH,
Jeff