About memory fragmentation and MallocNextGen

[blackwater]

Hi,

I have been developing a multithread application on C++ on HP-UX ia64 11.23 and 11.31. Usually it works as a daemon for at least a few days. The problem is that its virtual size measured by ps keeps growing while the application process requests.

I read carefully the thread discussing the similar question http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1351858 and I would like to ask a few more questions:

1) One of recommendation is to start using MallocNextGen. My question is whether this library is available for 64 applications or only for 32 applications.

2) Does MallocNextGen require setting any variable like _M_SBA_OPTS or not. I have not fine anything about that in http://www.docs.hp.com/en/5992-4174/5992-4174.pdf.

3) Then, how to get “Detailed Report” about memory? There was a long detailed report in the thread. I use info heap in gdb in order to get information about memory allocations but it only reports heap usage and list of allocations but I can’t find the “Detailed Report”. What am I doing wrong? The reason why I want to get a detailed report is to use this information for setting appropriately _M_SBA_OPTS.

P.S. this thread has been moved from HP-UX > General to HP-UX > languages - HP Forums Moderator

[Don Morris_1]

Yes, it has 32-bit and 64-bit libraries. Just link the appropriate one.

From the documentation, _M_SBA_OPTS has no influence with MallocNextGen.
(http://docs.hp.com/en/5992-4174/ch10s09.html -- the Compatibility section).

Don't know about the Detailed Report, sorry. I'm sure someone else will (likely Dennis).

[Dennis Handly]

>I read carefully the thread discussing the similar question

Have you determined whether you have a leak or fragmentation?

>how to get "Detailed Report" about memory?

Hmm, other than doing one by one, the documentation isn't obvious.
http://www.hp.com/go/wdb
Perhaps you have to write it to a file?
info heap filename
info heap idnumber
info heap arena
info heap arena [0 |1|2|..] blocks stacks
info heap process

You should also look at "Debugging dynamic memory usage errors using HP WDB".

[blackwater]

Dennis,

> Have you determined whether you have a leak or fragmentation?
I can't answer to your question. The problem is when I ran the command info leaks in order to get information about leaks I got:
(gdb) info leaks
Scanning for memory leaks...


Program received signal SIGTRAP, Trace/breakpoint trap
si_code: 1073741952 - .

The program being debugged was signaled while in a function called from GDB.
GDB has restored the context to what it was before the call.
To change this behavior use "set unwindonsignal off"
Evaluation of the expression containing the function (__rtc_leaks_info) will be abandoned.


OK, I did: set unwindonsignal off
Then I ran info leaks again and got:

Scanning for memory leaks...


Program received signal SIGTRAP, Trace/breakpoint trap
si_code: -2147151679 - .

The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on"
Evaluation of the expression containing the function (__rtc_leaks_info) will be abandoned.

[blackwater]

Dennis, you are right.
When I run: info leaks there is no detailed report.
When I run: info leaks filename there is a detailed report.

[Dennis Handly · Accepted solution]

>I can't answer to your question.

By calling mallinfo(3) you can. What about "info heap arena" and "info heap process"?

>The problem is when I ran the command info leaks in order to get info

What wdb version are you using? Have you downloaded 6.0?

>When I run: info leaks filename there is a detailed report.

Do two of them separated by time and see if it grows.

[blackwater]

Don, thanks for your answer.
Just want to clarify. If I use MallocNextGen all my statements like these:

ClassName1 *p1 = new ClassName1;
ClassName2 *p2 = new ClassName2[5];
int *p3 = new int[1000];
std::vector arr;
arr.push_back(ClassName3());

will use MallocNextGen?

[blackwater]

Dennis,
1)
My application is 64-bit. I read for example this thread: http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=943026. As far as I understand mallinfo doesn't return correct data for 64 applications. Am I right?

2)
As for gdb version:
srv2-rx8 B.11.31 ia64 bash-3.2$ gdb -v
HP gdb 6.0 for HP Itanium (32 or 64 bit) and target HP-UX 11iv2 and 11iv3.
Copyright 1986 - 2009 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 6.0 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.

And before running my application I did these:
set unwindonsignal on
set heap-check leaks on
set heap-check frame-count 16

[blackwater]

As for "info leaks". It doesn't always fail. When I run it while the application starts up it say:
(gdb) info leaks
Scanning for memory leaks...


2150464 bytes leaked in 3 blocks

No. Total bytes Blocks Address Function
0 2146304 1 0x600000000255c740 dtl::obj_allocator_base::allocate_new_block(unsigned long,unsigned long,char const*,char const*,unsigned long,bool)
1 4096 1 0xc098b000 _hp_ldap_client_connect_server()
2 64 1 0x600000000292fba0 snlui()



But after a thread processing requests has been started the "info leaks" gives me:
[Switching to thread 9 (system thread 7138095)]

Breakpoint 6, rating_server::tariffer::rate_current_record (
this=0x600000002fae9a50, id_rec=3) at rating_server.cpp:1626
1626 ++statjob.recs_total;
(gdb) info leaks
Scanning for memory leaks...


Program received signal SIGSEGV, Segmentation fault
si_code: 2 - SEGV_ACCERR - Invalid Permissions for object.
mark (first=0x600000000a1074b0, last=0x600000000a107510, leak_detection=true,
internal_gc=false, location=HEAP) at ../../../Src/gnu/gdb/infrtc.c:6825
6825 ../../../Src/gnu/gdb/infrtc.c: No such file or directory.
in ../../../Src/gnu/gdb/infrtc.c

arg sulong failed. 0, 0x9fffffffe97442a8

Setup args failed.

Pid 14862 was killed due to failure in writing to user register stack - possible stack overflow.

Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
No stack.

[Dennis Handly]

>will use MallocNextGen?

You tell us about your C++ application. If you haven't replaced operator new and operator new[] or added your own allocators, it will.

>As far as I understand mallinfo doesn't return correct data for 64 applications. Am I right?

It should return something but I recall they messed up and didn't expand to truly 64 bit values. But this may be enough to check for leaks.

>Pid 14862 was killed due to failure in writing to user register stack - possible stack overflow.

It appears you need to increase your thread stack size.

Instead of info leaks, you may want to try the info heap commands.