Heap Memory does not shrinks even when the memory is freed

[Rajesh Tiwari_1]

Hello,

I am working on C++ application on HP-UX11i. The application has a server process which allocates memory from heap at runtime and again frees it after completion of its task. since this is an server process it runs for long period of time i.e even for few weeks. As a result it keeps growing over the period even since thereis no memory leak.And
when it grows to some extent it becomes slow and boggy.

As i know that its an HP-UX OS feature that even when the memory is freed the heap does not shrinks. The freed memory is kept back to
process for future reuse.

Now my Question is there any HP patch available which actually result in shrinking of heap when memory is freed.?

Again, why my binary get slower when it becomes large in memory size as visible from top and glance plus?

Your inputs would be highly appreciated.

Thanks in advance.

Regards

[A. Clay Stephenson · Accepted solution]

This is normal memory allocation policy and it is not unique to HP-UX. It's a very typical UNIX method of handling "free()'ed" memory. The memory is made available for subsequent reuse by the same process but is not returned to the OS. There is a method that will return the memory BUT you must take over the full job of memory allocation; you cannot use malloc(), calloc(), realloc(), new() -- so this is probably not practical especially for C++. The key to doing this is the sbrk() system call. It's what underlies malloc(), new(), et al. The argument to sbrk can be positive (allocate memory) or negative (deallocate memory).

As to why your program execution slows when the process size is large. You wrote it; don't you know? Possible reasons include: 1) Longer times to search through larger data structures 2) Inefficient algorithms 3) Total system memory use may have grown so large that the system is now paging significantly.

[Rajesh Tiwari_1]

Thanks for the response Stephenson.

As u mentioned that one of the reason for slowness of the binary could be "the system paging significantly".

So, Is there any possibility that the process freed memory pages does pageout ??

why i m asking this question is my binary becomes very big after running for say 2-3 weeks and then becomes irreversible slow.When killed and started again, it becomes fast but that for a week and again becomes slow. So, this slowness can be attributed to the pageout even for the freed memory.


Thanks & Regards.

[A. Clay Stephenson]

Remember the OS doesn't have a clue (or care) that your dynamically allocated memory has been free()'ed. From the perspective of the OS, all free() has done is move memory from one internal (to the process) data structure to another. What is possible, it that TOTAL system memory use including your big process has grown so large that the system as a whole has to page. When your process gets sluggish, do a vmstat and look at the po (pageout) rates. If the pageout rate is high (above 10-20) for significant periods of time (minutes) then your system is definitely under memory pressure. You really should run tusc against your process when it is sluggish. You could then see what system calls are getting hammered.