cancel
Showing results for 
Search instead for 
Did you mean: 

Malloc (HP's memory handling)

Malloc (HP's memory handling)

Hi everyone,

I hope someone can help. Currently we have a Unibase application running out of memory every so often. When it is initially started, the memory usage is fine, but this builds up gradually over time until eventually the machine is out of memory. The APP is then stopped and restarted, memory grows etc..

Now, to me, this looks like a memory leak somewhere. The Unibase developers are adamant that this is an HPUX issue (see their comments attached).

Can anyone shed some more light before they convince us to buy more memory, which may ultimately be a waste?

Thanks!
11 REPLIES
James R. Ferguson
Acclaimed Contributor

Re: Malloc (HP's memory handling)

Hi:

A 'malloc()' allocates memory in the heap. A subsequent 'free()' returns that memory for reuse to the heap, but not to the system at-large. Upon termination of a process, the entire heap is returned to the system.

Regards!

...JRF...
Steven E. Protter
Exalted Contributor

Re: Malloc (HP's memory handling)

Shalom

http://www.hpux.ws/?p=8

Memory leak detector.

The response to a memory leak is to FIX it, not buy memory.

Once you show unibase evidence of a memory leak associated with one of their processes they'll have to fix it.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Dennis Handly
Acclaimed Contributor

Re: Malloc (HP's memory handling)

As JRF and Unibase says, the heap will grow and never return memory to the system.
So if your application uses lots of memory in the middle and frees most of it near the end, that application will need to exit.
Or be reprogrammed to use shared memory or mapped files.

Or you deal with this by adding lots of swap.
Do "swapinfo -tam" when you said "the machine is out of memory".

>The OS-process resources will not release (or decrease in size) after the processing completes

This isn't quite true. The VM use won't decrease but the memory should/could just be paged out.

Of course there still could be a leak in the application, check with wdb.
Duncan Edmonstone
Honored Contributor

Re: Malloc (HP's memory handling)

The statement from Unibase is broadly correct although it doesn't give much context.

Allocating memory is very expensive in CPU-time (as your UNIBASE person found when he tried using the other malloc library), so its generally preferable to leave the free'd memory in the processes data segment.

There are some fairly esoteric ways of tuning the behaviour of malloc if you're interested (look at the man page for malloc(3c) in the "EXTERNAL INFLUENCES" section), but I would be wary of using these without knowing exactly what you're doing. Better to request some assitance via the response centre for this (although it might be considered "consulting" and mean a cost!)

HTH

Duncan

HTH

Duncan
Dennis Handly
Acclaimed Contributor

Re: Malloc (HP's memory handling)

There could be N problems here:
1) An ordinary memory leak in the application.
2) Heap fragmentation, looks like a leak.
3) Swap space exhaustion because space is kept until program exit.

Since you said "memory grows", it probably isn't 3) like I first mentioned??
James R. Ferguson
Acclaimed Contributor

Re: Malloc (HP's memory handling)

Hi (again):

It occurs to me to also ask, do you ever issue 'kill -9' ? The point of the question is that your memory exhaustion could be at least partially a result of leaving shared memory segments lying around.

Regards!

...JRF...

Re: Malloc (HP's memory handling)

Hi all,

Thanks for the replies so far. Allow me to answer a few of your questions:

1.) Personally, I haven't used kill -9 on these machines, but it may be that someone else with the necessary access has...

2.) Swap space utilization was yesterday at 31%; today it's 43%. Still nowhere near fully utilized.

Quite frankly, I don't know how to chase a memory leak or even ordinary memory fragmentation in someone else's APP. And unfortunately, the APPS owners aren't too keen on assisting either as they want us to put in more memory.

I'll see what I can come up with in the next day or so and will then allocate points to all accordingly.

Marius

Dennis Handly
Acclaimed Contributor

Re: Malloc (HP's memory handling)

>I don't know how to chase a memory leak

Look at SEP's link for the simple/big picture.
For WDB:
http://www.hp.com/go/wdb
Go to the documentation link and then download:
» Debugging dynamic memory usage errors using HP WDB

>even ordinary memory fragmentation

You probably can't. :-(
Looks like a leak except libc/wdb/purify claims that there is no growth in allocations, just in free blocks.

>in someone else's APP.

You just need to know how to run wdb/gdb.

Re: Malloc (HP's memory handling)

Thanks to all who replied.

I have learned one or two things that I may use, but ultimately, it would depend on the co-operation of the Unibase people, especially if I want to go down the debugger -route.

Marius
Duncan Edmonstone
Honored Contributor

Re: Malloc (HP's memory handling)

marius,

you might also want to point the unibase folks at this new next gen malloc routine in the latest release of 11iv3 (11.31) - might be worth them looking at as well:

http://docs.hp.com/en/5992-4174/ch10s09.html

HTH

Duncan

HTH

Duncan
Dennis Handly
Acclaimed Contributor

Re: Malloc (HP's memory handling)

>especially if I want to go down the debugger route.

Actually if you know what you are doing, you don't need their help (unless stripped), unless you want them to change things, as Duncan suggests. :-)