Problem with 32bit vs 64bit Data and Stack Sizes

[Shawn D. Givler]

I am having a problem with our HP Itanium2 with 16Gb. The kernal settings for the 32bit and _64bit maxdsiz and maxssiz have been increased to their maximum (4 & 16Gb); however, ulimit reports that maxdsiz is only 4Gb because I am checking from a 32bit shell.
My 64bit engineering analysis software still seems to be running into the 4Gb datasiz limit. Models that require less than 4Gb run seemlessly. Models requiring more than 4Gb halt with a message that the software's request to allocate the additional memory beyond 4Gb have failed.

[A. Clay Stephenson]

I don't have an Itanium box to test on. The very first thing you should do is reduce your maxssiz values from their current ridiculous levels. Only extremely poorly written code ever need stacks larger than 64MB in 32-bit land and about 256MB in 64-bit land. Both of these are EXTREMELY generous. 32MB and 64MB respectively would still be generous values. Moreover, the stack does not limit dynamic memory allocation.

It would be helpful to know the exact errno value (generally the exit status of the program); I suspect that it is 27 - ENOMEM.

You should be able to duplicate this problem with a very small C program using calloc().

[Alex Lavrov.]

maxdsiz and maxssiz are for 32bit.
maxdsiz_64bit and maxssiz_64bit are for 64bit.

Different parameters, so it's not difference where from you check it, maxdsiz will stay 4GB and maxdsiz_64bit will be 16GB no matter from what shell you check it.

Are you sure that you software is 64bit?

what /path/to/binary
file /path/to/binary

Alex.

[Shawn D. Givler]

I have checked the executable and it definitely is 64bit.

%file xeq
xeq: ELF-64 executable object file - IA64

The executable runs on our other HP Itanium2 with 12Gb and the tunables are identical:

% kctune maxdsiz maxdsiz_64bit maxssiz maxssiz_64bit
Tunable Value Expression Changes
maxdsiz 4293918720 4293918720 Immed
maxdsiz_64bit 8589934592 8589934592 Immed
maxssiz 401604608 401604608 Immed
maxssiz_64bit 1073741824 1073741824 Immed

However, checking user limits shows that the problem machine (hp01) has a lower datasize limit:

hp02% ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 4193280
stack(kbytes) 392192
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 2048

hp01% ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 1696764
stack(kbytes) 392192
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 2048

Are these limits not the 32bit limits since I am checking them from a 32bit shell and correspond to the values reported by kctune?
How does one check what are a user's 64bit limits are and how does one increase them?

[Alex Lavrov.]

Right, I forgot about ulimit. Indeed you must increase ulimit too ... Think 10 points to yourself then :)

Read in the shell man about ulimit, how to do it. ("man sh", not "man ulimit", it won't work)


Alex.

[Don Morris_1]

Did you run ulimit from a *new* shell or an already running shell from before you made the tunable change?

The dynamic tunables take effect immediately but *only* for new processes. Limits for processes already running are already set. This also means that if your Engineering Analysis Software is already running - restart it.

Next, your stacks are way, way way too high for any reasonable belief -- and there's no way that you'll actually get all that stack and data. It is much more likely that the huge pre-reserved area of the stack is interfering with your data allocations, in fact.

Finally - if you want the system limits to be in force, don't set ulimit manually in the parent shell (and you may want to check whatever spawns your shells to make sure some profile script didn't already do this). The system gives processes the current system limits *if* the parent never changed their limits. Otherwise, the parent's limits are honored.