topic Re: 32 bit process terminates when data segement exceeds 64-bit data segement limit in Operating System - HP-UX

32 bit process terminates when data segement exceeds 64-bit data segement limit

Jimmy Tom_2 — Thu, 12 Jan 2006 17:10:13 GMT

I am having issues with my apps and it seems to have to do with the kernel parameter.

The kernel parameter that limits process data size for 32-bit processes is called maxdsize and is set to 2GB.
For 64-bit processes it is called maxdsize_64bit it is set to 1GB.
Our 32-bit process dies when it hits the 1GB limit.
(process also dies at 512MB data size when maxdsize_64bit is set to 512MB).

Our server is an rp3440 64 bit machine running hpux 11i The process is 32 bit compiled with +DA2.0 and linked with -N option to overcome the 1GB data limit.
(On former K-class servers the process normally went well beyond 1GB)

Any patches or parameter I need to set?
Thx in advance.

Re: 32 bit process terminates when data segement exceeds 64-bit data segement limit

Patrick Wallek — Thu, 12 Jan 2006 17:12:02 GMT

If the 64-bit parameter is set lower than the 32-bit parameter then the value of the 64-bit parameter will be used.

To solve this you should increase the maxdsiz_64 to be at least 2GB.

Re: 32 bit process terminates when data segement exceeds 64-bit data segement limit

A. Clay Stephenson — Thu, 12 Jan 2006 18:21:23 GMT

If any of the maxXsiz_64bit values are lower than their maxXsiz 32-bit counterparts, the smaller of the two wins. All of your xxx_64bit values should be at least as large as their 32bit cousins.

One of the strange things that happens is that ulimit always uses the smaller of the two values.

Re: 32 bit process terminates when data segement exceeds 64-bit data segement limit

Don Morris_1 — Thu, 12 Jan 2006 19:15:11 GMT

Not that strange once you know why.

When a 64-bit process forks off a 32-bit process, you have to make a decision somehow. The rlimit standard states that children should inherit the limits of their parents -- but the 32-bit maximum may be much lower than the 64-bit limit (even if the parent didn't change the limit). So the most restrictive of the limits is used (parent limit vs. 32-bit system-wide limit).

What happens here is that shells are almost always 32-bit processes... and are spawned from a 64-bit process (init). So init starts with the 64-bit (lower) limit, forks 32-bit children... which inherit the lower limit of the 64-bit parent instead of the system-wide 32-bit limit. All processes spawned from the shell inherit the limit of their parent (the shell)... so it propagates from there.

Re: 32 bit process terminates when data segement exceeds 64-bit data segement limit

A. Clay Stephenson — Thu, 12 Jan 2006 20:47:23 GMT

It's not that obvious to me but I'm dumb. This problem is when the 64-bit values are less than the 32-bit values and yet the 32-bit executables are limited by the smaller 64-bit values which should be ignored by a 32-bit executable --- including the shell.