chatr +q3p enable does not seem to be working

[Prashant_24]

Hi,
I have a strange problem with enabling 3rd quadrant private data space for a process.

1. maxdsiz is set to 2GB
2. chatr +q3p enable ovet_disco succeeds and shows that 3rd quadrant private space is enabled( By running chatr command)
-----
third quadrant private data space enabled
fourth quadrant private data space disabled
third quadrant global data space disabled
-------

Even after the above settings, the process crashes when it reaches 1GB. Log file shows the following.
Assertion failed: !(tmp == 0), file /opt/aCC/include/memory, line 493
ovet_disco died on signal: SIGABRT (6) (see /var/opt/OV/log/ovet_disco_err.log

Any ideas/suggestions will be greatly appreciated.

Thanks,
Prashant

[Don Morris_1 · Accepted solution]

And... architecture? OS? 32-bit or 64-bit process? maxssiz / maxssiz_64bit settings?

There's a lot of required information you're leaving out here.

Most likely possibilities would be:

1) Your (or whoever is running this) shell has a ulimit setting. This translates to a soft rlimit below the system defined hard limit of the tunable. Such limits are always propagated to the child process (that's the standard), hence your program gets the lower limit that was set in the shell in the first place.

2) You also raised maxssiz to something nutty like nigh-unto-1Gb. You can't get more than 1Gb in the process because the other 1Gb is reserved for the stack.

3) Similarly, besides the malloc -- your process is also doing private mmaps or other object creation. Your private space is 2Gb, but the other objects are preventing growing the heap itself beyond 1Gb. MAP_FIXED hints could be fragmenting the private address space as well.

4) Your problem isn't even with the rlimit per se -- but data corruption or pointer truncation. Just because tmp is NULL doesn't mean that an allocation failed -- did malloc/new explicitly fail? Is there an error?

5) You've hit some fun issue with 2Gb vs. the "magic" value for Infinity of (2Gb-1) in the standard and something got very confused. maxdsiz of 2.5Gb might be a better option.

The simplest first step (imho) would be to write a simple program to call getrlimit() and report the RLIM_DATA limits. Run it in the same environment as the prior program, and make sure you see the 2Gb limit you expect.

[Dennis Handly]

>Don: Just because tmp is NULL doesn't mean that an allocation failed

That's exactly what it means.

>The simplest first step would be to write a simple program to call getrlimit() and report the RLIM_DATA limits.

The shell's "ulimit -a" would show this.

[Don Morris_1]

Usually, yes -- but we don't know if there's memory corruption affecting the library.

And similarly, I don't trust the shell to report accurately in all cases [who knows what shell? Or if it is homegrown? Or if it just reports "Infinity" for anything over a signed 32-bit int? etc.]

Hence, let's validate the raw rlimit situation and work from there.

[Dennis Handly]

>Don: we don't know if there's memory corruption affecting the library.

Memory corruption usually causes a signal in libc's heap returns, not an error return.

[Prashant_24]

Hi,
Thanks very much for your responses.
It is 32 bit program running on HP-UX 11.11
maxssiz is about 8MB.
It does not seem to be due to memory corruption as it crashes exactly at 1GB everytime. I will check ulimit output and provide you soon.

Thanks & Regards,
Prashant