debugging very large applications

[Thomas Matelich]

(related to http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=982960 same application, trying to debug it under gdb or wdb. I started building my app dynamically though, removing the problem with mmap)

Really, I believe this issue comes down to kernel parameters somehow. I have 2GB memory and originally had 1 GB swap. Since I started working on this problem I have added 5 more GB of swap.

Here are some kernel param values I think are related to the problem. All of these were smaller at some point in my exploration:

maxdsiz - 0xCC000000
maxssiz - 0x4000000
maxtsiz - 0x40000000
maxswapchunks - 5000

When I run gdb and watch top, I see its memory usage grow to where SIZE is reported around 900-950 MB and RES is around 900 MB. gdb then fails with a message about virtual memory exhausted: can't allocate 95743917 bytes. WDB has the same behavior. I tried to run DDE but it won't come up for some reason.

While gdb is still running, using the memory it has used so far, /etc/swapinfo -t reports:

Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1048576 0 1048576 0% 0 - 1 /dev/vg00/lvol2
dev 5242880 113056 5129824 2% 0 - 0 /dev/vg00/lvol10
reserve - 1360172 -1360172
memory 1566484 465180 1101304 30%
total 7857940 1938408 5919532 25% - 0 -

Is there a trick to exceeding 1 GB memory in an application? Thanks!

Thanks,
Tom

[Kent Ostby]

If this is 11.23, then you may be running into a new bug. There is no official patch for the problem (coming sometime in 2006).

I would suggest you log a SW case with HP and ask them if they have a temporary patch that you can get.

[James R. Ferguson · Accepted solution]

Hi Tom:

You could try a 'chatr' on your executable to increase the third or fourth guadrant space. See the man pages for 'chatr'.

Regards!

...JRF...

[Thomas Matelich]

Thank you! The chatr q3p/q4p thing did the trick!

[Bill Hassell]

32bit applications are mapped into 4 quadrants of 1Gb each. So a program compiled without EXEC_MAGIC has it's data area stuck in one quadrant, thus the apx. 950Mb limit. Compile with -Wl -N to allow the data area to occupy most of two quadrants. +q3 and +q4 bunmp the limits up to about 2700 and 3700 megs resepctively. In all cases, each quadrant extension comes with several limits and considerations. Be sure to read the memory management white paper for 32bit programs.

Change the compilation to 64bits and now you can grab dozens of Gb of data area. Attached is a simple program to illustrate all the options.