ulimits, max sizes, and out of memory errors: 32bit

[Peter Heinemann]

Been through most of the postings on this subject but I'm still puzzled as to why a 32bit application is failing with "Failed to allocate memory (out of virtual memory)"

ulimits:

time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 4294575040
stack(kbytes) 392192
memory(kbytes) unlimited
coredump(blocks) unlimited

Corresponding to these kernel params (last figure is decimal value for the hex)

maxdsiz 0xf0000000 - 0xf0000000 4,026,531,840
maxdsiz_64bit 0xf0000000 - 0xf0000000 4,026,531,840
maxssiz 0x17F00000 - 0x17F00000 401,604,608
maxssiz_64bit 0x17F00000 - 0x17F00000 401,604,608



The Glance Process Memory Regions for the failed process looked like this when it died:

Memory Regions PID: 18484, pmdtm PPID: 12447 euid: 307 User: poweradm

Type RefCt RSS VSS Locked File Name
--------------------------------------------------------------------------------
DATA /Priv 1 1.60gb 1.60gb 0kb
MEMMAP/Priv 1 8kb 8kb 0kb /opt/.../server/pmdtm
MEMMAP/Priv 1 20kb 20kb 0kb
MEMMAP/Priv 1 8kb 8kb 0kb
MEMMAP/Priv 1 8kb 8kb 0kb /.../libpmdppivoter.sl
MEMMAP/Priv 1 12kb 12kb 0kb
MEMMAP/Priv 1 8kb 8kb 0kb /opt/.../libpmdpsvrcmn.sl
MEMMAP/Priv 1 36kb 36kb 0kb

Text RSS/VSS: 0kb/ 0kb Data RSS/VSS:1.6gb/1.6gb Stack RSS/VSS: 48kb/ 48kb
Shmem RSS/VSS:212kb/1.9mb Other RSS/VSS: 38mb/ 65mb

So:
If the maxdsiz for 32 bit O/S maximum is ~2gb (according to SAM help), where did the process hit the limit, or what limit was hit?

The application is 3rd party, not home-grown, so we can't modify it to 64bit ourselves.

My understanding from the HP technical paper and other postings (especially A.Clay) is that since my _64 bit kernel params are higher than their corresponding 32bit params they're not interfering.

Thanks for insights...

[A. Clay Stephenson]

Okay, you have several things going on. First, reduce maxdsiz from it's ridiculous value to 1GB or so; unless you enable some options to use other quadrants for data then 1GB is your limit. If you want to go beyond that then man chatr. I suspect that your real problem (since 32-bit applications have always had to respect the 1GB quadrant is that your maxssiz is too big. Because the stack and data come from the same quadrant with your stack set to about 400MB all that is left is 600MB regardless of the actual maxdsiz value. In almost all cases, 64MB is a very generous setting for maxssiz.

You could also be running out of swap space but I rather doubt it.

[Bill Hassell]

The MAXIMUM amount of memory that a 32 bit program can request is 960 megs. If your 3rd party provider did not compile the program for EXEC_MAGIC, that's it. Find another vendor. If the vendor needs some help, for a c program, use the flags: -Ae -Wl -N and now the program can address up to 1750 megs. Now if your program can indeed get in the 1500-1700 megs range, you can get more yourself *IF* you're running 11.00 with the latest patches or 11.11. Use the chatr command to change your program to use quadrant 3 and now it can grab up to 2700 megs:

chatr +q3p enable
WIth 11.11 (only) you can also enable +q4p and get 3700 megs with a 32bit program.

[Peter Heinemann]

Ok, saw your responses; still was puzzled; pulled out the Cooper/Moore internals book and dug in.

This system is 11.11 (apologies for not noting that previously).

The Quadrant limits are clearer, now, but:

1. Based on the Glance data, is it reasonable to assume that the application was compiled with EXEC_MAGIC? Isn't that the only way it could have Data RSS at 1.6gb?

2. If so, how does this change the stack vs. data ratio? 32-bit Text+Data+Stack < 2gb?

3. If not EXEC_MAGIC (or any other MAGIC) is the previous formula then 32-bit Text+Data+Stack < 1gb?

4. Anyone have a link to a white paper re: 32bit application memory management?

At this point, I'd like to optimize the 32bit settings to prove that there's no more to be done for this situation until management forks out some $ for the 64 bit version (no, I don't know why 32bit was purchased & installed...ancient history)

Points forthcoming. Thanks.

[A. Clay Stephenson · Accepted solution]

Let's assume nothing but instead ask the executable itself.

chatr my.exe

If EXEC_MAGIC was enabled then you will
see "normal executable"; if not "shared executable". If an EXEC_MAGIC executable has been converted to SHMEM_MAGIC then you will see "normal SHMEM_MAGIC executable". As mentioned by Bill, unless your developer's originally compiled with "-N" the executable cannot be either EXEC_MAGIC or SHARED_MAGIC. Only a program originally linked as EXEC_MAGIC can be converted to SHMEM_MAGIC.

Your stack is still too large because even when EXEC_MAGIC is in play the same rules still apply because stack and data still coexist in at least one quadrant so whatever is set aside for the stack segment is not available for the data segment. The stack is really only used to hold temporary variables (variables local to functions and the function stack frames) so unless your code is very poorly written, 64MB is extremely generous and 32MB should be more than adequate. Even 64-bit programs don't need large stacks (unless poorly written) for exactly the same reason.

[Bill Hassell]

As Clay said, the file command and the chatr command will tell you about the program's type. But since Glance reports a Resident Set Siz of 1.6Gb, yes, it is EXEC_MAGIC. The max stack size is way, way too large and it indeed reduces the available quadrant space. You can change the flag for q3p (use the 3rd quadrant for local data). Note: if your program also uses shared memory, this is not an option. Look at ipcs -bmop when your program is running. +q3p can be enable or disabled using chatr. It simply changes a flag in the excutable.

[Peter Heinemann]

Clay, Bill,

Yes, the program erroring was in fact done with EXEC_MAGIC:
# chatr /opt/etl/informatica/pc/pmdtm
/opt/etl/informatica/pc/pmdtm:
normal executable
shared library dynamic path search:


but given that
1. the erroring program is called by a parent which was NOT compiled with EXEC_MAGIC
and
2. the whole shootin' match is 3rd party

there's not much else to do. I will rationalize the max?siz parameters.

[A. Clay Stephenson]

The bitness of the parent process doesn't really matter. Reduce your maxdsiz to 2GB or a little less but mainly reduce maxssiz and that should allow a bigger dynamic memory allocation -- of another few hundred MB.