- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: About memory fragmentation and MallocNextGen
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-31-2009 07:26 AM - last edited on 11-20-2012 07:36 PM by Maiko-I
07-31-2009 07:26 AM - last edited on 11-20-2012 07:36 PM by Maiko-I
Hi,
I have been developing a multithread application on C++ on HP-UX ia64 11.23 and 11.31. Usually it works as a daemon for at least a few days. The problem is that its virtual size measured by ps keeps growing while the application process requests.
I read carefully the thread discussing the similar question http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1351858 and I would like to ask a few more questions:
1) One of recommendation is to start using MallocNextGen. My question is whether this library is available for 64 applications or only for 32 applications.
2) Does MallocNextGen require setting any variable like _M_SBA_OPTS or not. I have not fine anything about that in http://www.docs.hp.com/en/5992-4174/5992-4174.pdf.
3) Then, how to get “Detailed Report” about memory? There was a long detailed report in the thread. I use info heap in gdb in order to get information about memory allocations but it only reports heap usage and list of allocations but I can’t find the “Detailed Report”. What am I doing wrong? The reason why I want to get a detailed report is to use this information for setting appropriately _M_SBA_OPTS.
P.S. this thread has been moved from HP-UX > General to HP-UX > languages - HP Forums Moderator
Solved! Go to Solution.
- Tags:
- malloc
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-31-2009 07:47 AM
07-31-2009 07:47 AM
Re: About memory fragmentation and MallocNextGen
From the documentation, _M_SBA_OPTS has no influence with MallocNextGen.
(http://docs.hp.com/en/5992-4174/ch10s09.html -- the Compatibility section).
Don't know about the Detailed Report, sorry. I'm sure someone else will (likely Dennis).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-31-2009 10:40 AM
07-31-2009 10:40 AM
Re: About memory fragmentation and MallocNextGen
Have you determined whether you have a leak or fragmentation?
>how to get "Detailed Report" about memory?
Hmm, other than doing one by one, the documentation isn't obvious.
http://www.hp.com/go/wdb
Perhaps you have to write it to a file?
info heap filename
info heap idnumber
info heap arena
info heap arena [0 |1|2|..] blocks stacks
info heap process
You should also look at "Debugging dynamic memory usage errors using HP WDB".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 12:03 AM
08-03-2009 12:03 AM
Re: About memory fragmentation and MallocNextGen
> Have you determined whether you have a leak or fragmentation?
I can't answer to your question. The problem is when I ran the command info leaks in order to get information about leaks I got:
(gdb) info leaks
Scanning for memory leaks...
Program received signal SIGTRAP, Trace/breakpoint trap
si_code: 1073741952 - .
The program being debugged was signaled while in a function called from GDB.
GDB has restored the context to what it was before the call.
To change this behavior use "set unwindonsignal off"
Evaluation of the expression containing the function (__rtc_leaks_info) will be abandoned.
OK, I did: set unwindonsignal off
Then I ran info leaks again and got:
Scanning for memory leaks...
Program received signal SIGTRAP, Trace/breakpoint trap
si_code: -2147151679 - .
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on"
Evaluation of the expression containing the function (__rtc_leaks_info) will be abandoned.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 12:05 AM
08-03-2009 12:05 AM
Re: About memory fragmentation and MallocNextGen
When I run: info leaks there is no detailed report.
When I run: info leaks filename there is a detailed report.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 12:25 AM
08-03-2009 12:25 AM
SolutionBy calling mallinfo(3) you can. What about "info heap arena" and "info heap process"?
>The problem is when I ran the command info leaks in order to get info
What wdb version are you using? Have you downloaded 6.0?
>When I run: info leaks filename there is a detailed report.
Do two of them separated by time and see if it grows.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 12:41 AM
08-03-2009 12:41 AM
Re: About memory fragmentation and MallocNextGen
Just want to clarify. If I use MallocNextGen all my statements like these:
ClassName1 *p1 = new ClassName1;
ClassName2 *p2 = new ClassName2[5];
int *p3 = new int[1000];
std::vector
arr.push_back(ClassName3());
will use MallocNextGen?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 12:47 AM
08-03-2009 12:47 AM
Re: About memory fragmentation and MallocNextGen
1)
My application is 64-bit. I read for example this thread: http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=943026. As far as I understand mallinfo doesn't return correct data for 64 applications. Am I right?
2)
As for gdb version:
srv2-rx8 B.11.31 ia64 bash-3.2$ gdb -v
HP gdb 6.0 for HP Itanium (32 or 64 bit) and target HP-UX 11iv2 and 11iv3.
Copyright 1986 - 2009 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 6.0 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
And before running my application I did these:
set unwindonsignal on
set heap-check leaks on
set heap-check frame-count 16
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 01:06 AM
08-03-2009 01:06 AM
Re: About memory fragmentation and MallocNextGen
(gdb) info leaks
Scanning for memory leaks...
2150464 bytes leaked in 3 blocks
No. Total bytes Blocks Address Function
0 2146304 1 0x600000000255c740 dtl::obj_allocator_base::allocate_new_block(unsigned long,unsigned long,char const*,char const*,unsigned long,bool)
1 4096 1 0xc098b000 _hp_ldap_client_connect_server()
2 64 1 0x600000000292fba0 snlui()
But after a thread processing requests has been started the "info leaks" gives me:
[Switching to thread 9 (system thread 7138095)]
Breakpoint 6, rating_server::tariffer::rate_current_record (
this=0x600000002fae9a50, id_rec=3) at rating_server.cpp:1626
1626 ++statjob.recs_total;
(gdb) info leaks
Scanning for memory leaks...
Program received signal SIGSEGV, Segmentation fault
si_code: 2 - SEGV_ACCERR - Invalid Permissions for object.
mark (first=0x600000000a1074b0, last=0x600000000a107510, leak_detection=true,
internal_gc=false, location=HEAP) at ../../../Src/gnu/gdb/infrtc.c:6825
6825 ../../../Src/gnu/gdb/infrtc.c: No such file or directory.
in ../../../Src/gnu/gdb/infrtc.c
arg sulong failed. 0, 0x9fffffffe97442a8
Setup args failed.
Pid 14862 was killed due to failure in writing to user register stack - possible stack overflow.
Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
No stack.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 01:24 AM
08-03-2009 01:24 AM
Re: About memory fragmentation and MallocNextGen
You tell us about your C++ application. If you haven't replaced operator new and operator new[] or added your own allocators, it will.
>As far as I understand mallinfo doesn't return correct data for 64 applications. Am I right?
It should return something but I recall they messed up and didn't expand to truly 64 bit values. But this may be enough to check for leaks.
>Pid 14862 was killed due to failure in writing to user register stack - possible stack overflow.
It appears you need to increase your thread stack size.
Instead of info leaks, you may want to try the info heap commands.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2009 11:38 PM
08-03-2009 11:38 PM
Re: About memory fragmentation and MallocNextGen
I have already used info heap filename and then compared the reports and found a memory leak. I think formally speaking iIt's not a memory leak, but anyway, the result was as if I had a memoty leak.
The problem was with a global list variable and a global boolean flag. They were defined and used in this way:
std::list
bool global_list_variable_initialized = false;
void f()
{
if(!global_list_variable_initialized) {
global_list_variable.push_back(MyClass());
// global_list_variable_initialized
// by mistake was not set to true
}
}
As for your suggestion "It appears you need to increase your thread stack size.". What do you mean? My application doesn't cause this error when it runs. It only happens when I run info leaks under gdb. So where should I increase thread stack size and how? Do you mean setting maxssiz_64bit? Could you explain, please?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2009 01:59 AM
08-04-2009 01:59 AM
Re: About memory fragmentation and MallocNextGen
Why have two data structures to track one thing? Just use:
if (global_list_variable.begin() == global_list_variable.end()) {
global_list_variable.push_back(MyClass())
(Unless f() is called after there are entries on the list.)
My application doesn't cause this error when it runs. It only happens when I run info leaks under gdb.
Right. IF you want to use gdb, you would need to increase your thread stacksize.
>So where should I increase thread stack size and how?
When you do pthread_create, you need to set the thread stacksize attribute with:
pthread_attr_setstacksize(3)
Or export PTHREAD_DEFAULT_STACK_SIZE with a larger size, in bytes.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2009 07:50 AM
08-06-2009 07:50 AM
Re: About memory fragmentation and MallocNextGen
I made a search and found in one of libraries that I use a call to pthread_attr_setstacksize. I think I will comment it and recompile my application in order to experiment with PTHREAD_DEFAULT_STACK_SIZE. If I still have a problem with info leaks causing a coredump I will start another thread later.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2009 08:23 AM
08-06-2009 08:23 AM
Re: About memory fragmentation and MallocNextGen
2) Seems like there is no need to set any enviroment variable in order to use it and existing enviroment variable like _M_SBA_OPTS doen't affect it.
3) The detailed memory report is generated
if I run a command "info heap filename"
4) "info heap" provides lots of information in order to find memory leaks and situations which look like a memory leak.
5) In order to run "info leak" for my application I need to increase thread stack size but I can't do it right now just by setting PTHREAD_DEFAULT_STACK_SIZE because in one of my library there is a call to pthread_attr_setstacksize(). I need to comment the call and recompile the application in order to see if "info leaks" does not result in a coredump.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2009 08:39 AM
08-06-2009 08:39 AM
Re: About memory fragmentation and MallocNextGen
The detailed report showed that the majority of memory blocks are less than 1024 bytes and there are few which are quite big.
So it is appropriate to set
_M_SBA_OPTS=1024:100:16
Seems like after assigning _M_SBA_OPTS this value I can expect that there might be no memory fragmentation.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-07-2009 02:47 AM
08-07-2009 02:47 AM
Re: About memory fragmentation and MallocNextGen
Here it looks like your main thread has exited when 'info leaks' fails with stack overflow. Could you please check if this is the case? If so, could you check if the main thread can be retained till whenever you need to issue the command to detect leaks?
- Jini.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2009 11:59 PM
09-22-2009 11:59 PM
Re: About memory fragmentation and MallocNextGen
At this point I decided to change the way I test the program. Instead of loading all the records from a database I changed a configuration of the program and started loading only records which were necessary for running my tests. That resulted in a significant decrease in number of memory allocations. And as a result of it executing "info leaks" didn't result in a coredump.