Malloc (HP's memory handling)

[marius etsebeth_1]

Hi everyone,

I hope someone can help. Currently we have a Unibase application running out of memory every so often. When it is initially started, the memory usage is fine, but this builds up gradually over time until eventually the machine is out of memory. The APP is then stopped and restarted, memory grows etc..

Now, to me, this looks like a memory leak somewhere. The Unibase developers are adamant that this is an HPUX issue (see their comments attached).

Can anyone shed some more light before they convince us to buy more memory, which may ultimately be a waste?

Thanks!

[James R. Ferguson]

Hi:

A 'malloc()' allocates memory in the heap. A subsequent 'free()' returns that memory for reuse to the heap, but not to the system at-large. Upon termination of a process, the entire heap is returned to the system.

Regards!

...JRF...

[Steven E. Protter]

Shalom

http://www.hpux.ws/?p=8

Memory leak detector.

The response to a memory leak is to FIX it, not buy memory.

Once you show unibase evidence of a memory leak associated with one of their processes they'll have to fix it.

SEP

[Dennis Handly]

As JRF and Unibase says, the heap will grow and never return memory to the system.
So if your application uses lots of memory in the middle and frees most of it near the end, that application will need to exit.
Or be reprogrammed to use shared memory or mapped files.

Or you deal with this by adding lots of swap.
Do "swapinfo -tam" when you said "the machine is out of memory".

>The OS-process resources will not release (or decrease in size) after the processing completes

This isn't quite true. The VM use won't decrease but the memory should/could just be paged out.

Of course there still could be a leak in the application, check with wdb.

[Duncan Edmonstone]

The statement from Unibase is broadly correct although it doesn't give much context.

Allocating memory is very expensive in CPU-time (as your UNIBASE person found when he tried using the other malloc library), so its generally preferable to leave the free'd memory in the processes data segment.

There are some fairly esoteric ways of tuning the behaviour of malloc if you're interested (look at the man page for malloc(3c) in the "EXTERNAL INFLUENCES" section), but I would be wary of using these without knowing exactly what you're doing. Better to request some assitance via the response centre for this (although it might be considered "consulting" and mean a cost!)

HTH

Duncan

[Dennis Handly]

There could be N problems here:
1) An ordinary memory leak in the application.
2) Heap fragmentation, looks like a leak.
3) Swap space exhaustion because space is kept until program exit.

Since you said "memory grows", it probably isn't 3) like I first mentioned??

[James R. Ferguson]

Hi (again):

It occurs to me to also ask, do you ever issue 'kill -9' ? The point of the question is that your memory exhaustion could be at least partially a result of leaving shared memory segments lying around.

Regards!

...JRF...

[marius etsebeth_1]

Hi all,

Thanks for the replies so far. Allow me to answer a few of your questions:

1.) Personally, I haven't used kill -9 on these machines, but it may be that someone else with the necessary access has...

2.) Swap space utilization was yesterday at 31%; today it's 43%. Still nowhere near fully utilized.

Quite frankly, I don't know how to chase a memory leak or even ordinary memory fragmentation in someone else's APP. And unfortunately, the APPS owners aren't too keen on assisting either as they want us to put in more memory.

I'll see what I can come up with in the next day or so and will then allocate points to all accordingly.

Marius

[Dennis Handly]

>I don't know how to chase a memory leak

Look at SEP's link for the simple/big picture.
For WDB:
http://www.hp.com/go/wdb
Go to the documentation link and then download:
» Debugging dynamic memory usage errors using HP WDB

>even ordinary memory fragmentation

You probably can't. :-(
Looks like a leak except libc/wdb/purify claims that there is no growth in allocations, just in free blocks.

>in someone else's APP.

You just need to know how to run wdb/gdb.

[marius etsebeth_1]

Thanks to all who replied.

I have learned one or two things that I may use, but ultimately, it would depend on the co-operation of the Unibase people, especially if I want to go down the debugger -route.

Marius