free() not really freeing?

[Deepak Extross]

Greetings,

I've noticed that on HP11.00, a call to free() to release previously malloc'ed memory does not relly release the memory altogether.
The memory usage (as viewed with 'glance' or just 'top') does not reduce after a call to free().

I've also found that if I malloc and free say 1000 bytes in a continuous loop, the average memory used remains static. But if I change the chunk size grabbed (say reduce it by 10 bytes in each iteration), then the memeory usage keeps increasing, very much like a memory leak.

It looks like free() isnt forcing the OS to release the memory back to the system, but merely recommending that it does so. And obviously, the OS has other plans :-)
Is there any way I can work around this?

[A. Clay Stephenson · Accepted solution]

Hi:

Free() is actually working as intended. It does not return memory to the OS but it does push the memory block back onto the calling process'es heap for use by a subsequent call to malloc, calloc, or realloc.

You can use the brk() and sbrk() system calls to do what you are trying to do but in that case you MUST NOT use the malloc calls or chaos is guaranteed. You must also make sure that any functions you use do not call malloc either. For these reasons, using brk() is seldom practical.

Plan B. Use shared memory using IPC_PRIVATE keys and detach and remove the shared memory segment when finished with it.



Regards, Clay

[Deepak Extross]

Thanks, Clay..that explains it.

But if I may carry this thread a little further, isn't this behavior a trifle inconvenient? Consider this case:
My process gets some input - files, message queues, whatever. The size of the input message is not fixed. For each input message, I determine its length, malloc()space to hold it, read it into the space, process it and then free() it.
Now, if the OS does not release the memory completely once I free() it, won't it cause the same problems as a memory leak would?

Besides, if I malloc 100 bytes, free it and then malloc 90, I find that the OS grabs a fresh chunk of memory instead of re-using the old chunk. Defeats the purpose, somewhat.

[A. Clay Stephenson]

Hi again,

Well it may be inconvenient but that's the way it works. One thing to point out is that the memory allocations in your examples are quite small and most of the tools only have a resolution down to a page (4k) of memory. Malloc is able to coalesce small freed chunks into a larger chunk but you are not going to see this in only a few memory allocations.

This is not really a memory leak in the sense that soon a process like you describe reaches equilibrium so that no new memory has to be requested from the OS but is simply reclaimed from the heap. What you don't know is that a tradeoff was made in the memory allocation routines. The first memory allocation actually requests more than the malloc called for but subsequent requests are then doled out from this 'larger than life' chunk. Only when this chunk is used up or more memory is required than is left does a new call go to sbrk to get memory from the OS. There used to be a difference between malloc(3c) and malloc(3x) but now the type have been merged and the faster but potentially more wasteful 3x version is used.

[Gregory Fruth]

Very few (if any) C implementations really return
memory to the OS when free() is called. Most
implementations just return the memory to the
pool that malloc() maintains for itself using
brk() and sbrk(). The man page says:

free() Deallocates the space pointed to by ptr (a
pointer to a block previously allocated by
malloc(), realloc(), or calloc()) and makes the
space available for further allocation.

I heard somewhere that Tru64 (formerly Digital Unix,
formerly OSF/1) does actually return the memory to
the OS, but I can't verify this.

HTH