Address space of 64bit executables on 11i v2 for Itanium

[dscao]

When started from a csh, all 64bit executables has its private space limited by the csh "limit datasize". But this "datasize" is derived from the kernel parameter "maxdsiz" which is supposed to be for 32 bit app.
I think this is a bug, but is there any solution? POSIX sh (/usr/bin/sh) does not have such issue, because the "data size" is unlimited.

[A. Clay Stephenson]

There is a maxdsiz_64bit tunable as well and this determines the maximum data segment size of 64-bit processes. Now the "gotcha" is that maxdsiz_64bit must be greater than maxdsiz. There are also 64-bit cousins for maxssiz and maxtsiz. Don't go nuts on maxssiz because only very poorly written code would ever require a stack larger than 64MiB or so and that is extremely generous even for a 64-bit program.

[dscao]

I know the "maxdsiz_64bit". In this case, maxdsiz is 2GB, maxdsiz_64bit is 4GB, but the 64bit app can only allocate no more than 2GB.
The csh "limit datasize" is 2GB, it can not be raised to 4GB. I guess it is limited by the "maxdsiz".

[Dennis Handly]

I have no problems allocating 4 Gb in a 64 bit applications when using the scummy C shell.
$ csh
% ulimit -a
data(kbytes) 2015464
% a.out
malloc returned 6000000000004030

maxdsiz 0x7b03a000 0x7B03A000
maxdsiz_64bit 0x4000000000 0x4000000000

>I guess it is limited by the "maxdsiz".

It probably only prints the maxdsiz value.
I don't see getrlimit64/setrlimit64 being called in any shell.

[dscao]

I can not reproduce the issue in my boxes where it is HP-UX v2 on a PA-RISC. The issue can be reproduced on the customer site. The issue goes away when the login shell is /usr/bin/sh. So I guess it's a C shell issue. It is impossible to get access to the boxes of the customer remotely. I can get more information if it is needed. I just only to know if someone knows the C shell issue and how to resolve it.

[Don Morris_1 · Accepted solution]

Note that the POSIX rlimit standard (which is what's coming into play here) requires child processes of a parent which modifies their limit to inherit the changed limits.

What that means here is that if the shell is *explicitly* changing the limit, the system default no longer applies -- the 64-bit child process inherits the limit of the shell. Check to make sure the customer hasn't got a .profile statement or other means that invokes a limit in the 32-bit shell which causes that lower limit to propagate to the child.

[dscao]

Yes. The user modified limit (datasize) will have effect on any child from this shell, whether in /usr/bin/sh shell or /usr/bin/csh shell, whether the child is a 64bit or 32bit application.

[dscao]

Thanks to Morris.