Memory leak[HP-UX] - Increasing heap size.

[Pratheesh]

We have a memory leak problem faced in a C++ program compiled in aCC and standard linker.

Symptom: The performance of the exe reduces after sometime and seing the memory occuppancy by top command increases over a period of time while the process is running as:

SIZE RES
50436K 21656K <-- After processing 100recs
50564K 21784K <-- After processing 300recs

We tried with gdb5.7 to trace this leak and gdb didn't detect anything. In the sameway we have ported the code to another UNIX platform and used a tool there to identify without any luck.

While using gdb:
1) gdb> info leaks command at both the occassion didn't show any differential leaks.
2) gdb> info heap command shows the total heap size and outstanding allocations. The outstanding allocations shows same for both the occassions(with 100 records and subsequently 300 records for the same process)and Heap size is varying as:

a) With 100 records, the output is:
Info heap:

Analyzing heap ...
Actual Heap Usage:
Heap Start= 0x80000001000d68e8
Heap End = 0x8000000100f00000
Heap Size = 14849816 bytes
Outstanding Allocations:
1749246 bytes allocated in 3919 blocks

b) With 300 records, the output is:
Info heap:

Analyzing heap ...
Actual Heap Usage:
Heap Start = 0x80000001000d68e8
Heap End = 0x8000000100f20000
Heap Size = 14980888 bytes

Outstanding Allocations:
1749246 bytes allocated in 3919 blocks


This difference in heap size is keep on increasing eventhough gdb is not detecting any memory leak. The code is written so that independent of howmany of data you feed it should not result in additional memory usage.

Could you please help us in the follwoing questions?

Q1) Whats the difference between "Heap size" and "Outstanding allocations" in the info heap output?
Q2) If there are no changes in "Outstanding allocations" and if the "Heap size" is varying what should we concur? Also, gdb is not showing any memory leak.
Q3) Whats are other loop holes in code other that noraml malloc/free and new/delete will result in such a heap size growth?
Q4) Any other gdb options, Can we explore?

Thanks in advance for your help,

Pratheesh/Rajesh.

[Dennis Handly]

>This difference in heap size is keep on increasing even though gdb is not detecting any memory leak. The code is written so that independent of how many of data you feed it should not result in additional memory usage.

You either have fragmented the heap. Or if using threads, you may have some type of arena bug. Are you on the latest patches?

>Q1) What's the difference between "Heap size" and "Outstanding allocations" in the info heap output?

Heap size is the total space, including heap overhead.
Outstanding allocations is how much you have allocated.

>Q2) If there are no changes in "Outstanding allocations" and if the "Heap size" is varying what should we concur?

Heap fragmentation or libc bug. For the former, you need to make sure you don't allocate so many random sizes. Round up to some common size.

>Q3) What are other loopholes

Fragmentation.

[Pratheesh]

We tested this in three different machine (ALPHA, PA-RISC and ITANIUM).And found that the memory leak is happens only in Itanium Machine.

We used GDB in Itanium to find the memory leak. But fail to find any leak.

Could you please let us know do you have faced such behavior in any itanium environment ?

[Dennis Handly]

>found that the memory leak is happens only on Integrity. But fail to find any leak.

Then it is either fragmentation or some libc issue. If you checked for PA, did you run this on 11.23?

>Could you please let us know do you have faced such behavior in any Integrity environment?

I've seen fragmentation on PA and IPF. I've header about libc bugs when using threads.
So make sure you have the latest libc and libphread patches.

[Pratheesh]

Our application is running on 11.23 and it is single threaded.

Can you please let us know how we can investigate on the 'fragmentation' related issue ?

Thanks in advance for your help,
Pratheesh/Rajesh

[Dennis Handly]

>Our application is running on 11.23 and it is single threaded.

Ok, that leaves out the multiple arena issue.

>Can you please let us know how we can investigate on the 'fragmentation' related issue?

Other than make sure you have the latest libc patches, there isn't much you can do other that file a request to the Response Center and see if they have any suggestions.

Or as I suggested, make sure you round up your sizes to fewer random values. If your blocks are "small" you could try adjusting the "grain" in the Small Block Allocator, malloc(1), _M_SBA_OPTS.

I suppose you could use wdb's "set heap-check" and do "info heap" to see what your sizes are and how many.

[Rodney Myers]

- We tried with gdb5.7 to trace this leak
- and gdb didn't detect anything.

Why do you think your application is leaking memory?

It seems you expect free() to reduce the data segment of your process. It doesn't. free() makes the memory available for future allocation.

You could reduce the data segment of your process. See brk(). It's not practical though. Using brk() with malloc() will be guaranteed fun. And I doubt this will solve your performance problem.

[Dennis Handly]

>Rodney: Why do you think your application is leaking memory?

I suggested heap fragmentation.

>It seems you expect free() to reduce the data segment of your process.

Hmm. That could also be happening. I assumed that one record is processed at a time and not all saved up before freeing. Especially since it was mentioned:
The code is written so that independent of how many of data you feed it should not result in additional memory usage.