running 32 bit exe in 64 bit machine-Core dumps after 2.1 GB running process size

[Nagaraj komarsamy]

We are running an HP-UX 32-bit executable on 64 bit processor.

The executable is crossing the process running size of 2.1 GB.

It has dumping core at 1.7 GB.

Then as per the HP-UX manual we have enabled the 3rd quadrant by

chatr +q3p enable myexename

In the HP-UX manuals it has mentioned that this 3rd quadrant enabled apllications will use the memory upto 2.8 GB

But even after applying this +q3p option , our application is dumping core at 2.1 GB

We have applied chatr +q4p enable MyExeName, to enable 4th quadrant.
But even after applying this also, our application is dummping core at 2.1 GB(2104 MB).
Can anyone help us to figure out what is limiting our application to run beyond 2.1 GB

How can we make the application to run with accessing upto 2.8 GB of memory?

We are using HP-UX 11.00

The following are the system details

# swapinfo -ma

Mb Mb Mb PCT START/ Mb

TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME

dev 8192 85 8107 1% 0 - 1 /dev/vg00/lvol2

reserve - 2168 -2168

memory 3149 1515 1634 48%

# swapinfo -a

Kb Kb Kb PCT START/ Kb

TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME

dev 8388608 86612 8301996 1% 0 - 1 /dev/vg00/lvol2

reserve - 2220488 -2220488

memory 3224700 1551156 1673544 48%

#



# swapinfo -tm

Mb Mb Mb PCT START/ Mb

TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME

dev 8192 105 8087 1% 0 - 1 /dev/vg00/lvol2

reserve - 2164 -2164

memory 3149 1483 1666 47%

total 11341 3752 7589 33% - 0 -

#



kmtune output

--------------

max_mem_window 0

max_thread_proc 5000

maxdsiz 3221225472

maxdsiz_64bit 34359738368

maxfiles 200

maxfiles_lim 2048

maxqueuetime 0

maxssiz 167772160

maxssiz_64bit 167772160

maxswapchunks 10000

maxtsiz 0X40000000

maxtsiz_64bit 0X0000000400000000

maxuprc 75

maxusers 128

maxvgs 10

mesg 1

minfree 0



shmmax 0X4000000 (64 MB)



swapmem_on 1



# ulimit -a

time(seconds) unlimited

file(blocks) unlimited

data(kbytes) unlimited

stack(kbytes) 163840

memory(kbytes) unlimited

coredump(blocks) 4194303

nofiles(descriptors) 200

Please help us if you have any clue.

Regards,
Naga

[V. Nyga]

Hi,

are we talking about 2,1 GB memory or 2,1 GB file size?
If it's file size you have to enable the option largefiles for your file system.

Volkmar

[Hein van den Heuvel]

Looks like you are ready for a move to a real 64 bit version of the application!
Any chances of that happening?

>> shmmax 0X4000000 (64 MB)

That a little small for many applications, but you might not use any. Check with ipcs -m
while the process is (still) running.
I typically see 2GB or 60% of physical memory. Still, it is unlikely to be a factor.

Be sure to google for something like: +q3p +hpux +site:hp.com
or +site:itrc.hp.com
for prior discussions and hints.
For example:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=72276
and
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1113916

Good luck,
Hein.

[Nagaraj komarsamy]

Volkmar,

Thanks for your promt reply.
Here we talking about process memory size.
Our executable loading huge amount of data to memory.

When the process try to allocate memory beyond 2104 MB, calloc fails stating error code 12.

Regards,
Naga

[Heironimus]

Are you doing a series of small memory allocations or are you trying to do one really big allocation? I think I remember that even with q3/q4 enabled there are problems with allocating one chunk that's too large.

[Don Morris_1]

First thought -- did you set ulimit for data to "unlimited" yourself? unlimited to the shell really is RLIM_INFINITY from the getrlimit() system cal. And the 32-bit value for RLIM_INFINTY is (2Gb - 1) [signed 32-bit integer]. That's suspiciously close to your calloc() failure point. (And since you aren't completely clear if you fail trying to alloc 2104Mb or when you already have 2104Mb and are growing, I can't dismiss the possibility). So the first thing I'd try is from a shell where you haven't messed with the ulimit and inherit the defaults from the tunable. That isn't to say 11.0 may not have an issue somewhere with maxdsiz above the standard sentinel value for RLIM_INFINITY (but I can't find one in a defect search at the moment), but the possibility should be eliminated.

If that isn't the problem -- the next obvious step would be to look at the resulting core in a debugger or use pstat_getprocvm() on the running process to check the virtual object layout in the process. You may have several private mmap's [which even worse may try MAP_FIXED or something] which the heap runs into. Hard to say with only this data. Sample code to report virtual objects attached, but fair warning -- I wrote this on v3 and intended it more for v2 / v3 -- it may take a little tweaking to compile and run on 11.0 -- I don't have such a box to do so myself.

[Dennis Handly]

># swapinfo -ma

No need to run three swapinfo commands, just use "swapinfo -tam".

Do you have the last libc patch for 11.00?

>Don: the next obvious step would be to look at the resulting core in a debugger ... to check the virtual object layout in the process. You may have several private mmap's

In gdb on a core file you can use "info file" to get the layouts. Using adb:
adb - core
$m
$q

I'm not sure if malloc is smart enough to use mmap(2) to allocate space if brk(2) fails?