32 bit processes have different limits depending on machine

[Daniel Gowans]

I have an executable that resides on one J6000 and is shared over NFS with 4 other J6000s. On two of these machines, the process can grow up to about 1.7 GB. On the other 3, it can only grow to about 1.0 GB.

After that, they give memory allocation errors. This executable is a vendor application and can't be re-compiled, but I was told that it was compiled with shmem_magic, and all that good stuff so that it could take up to 1.7 GB (it is 32 bit).

Why are some of my machines still limiting it to 1 GB? They are all running HP-UX 11i. The big difference is that many kernel parameters are different on the 2 that will let it grow to 1.7 GB. Which kernel parameters do I need to change on the others to get them to all allow the process to reach its maximum size?

Thanks!

[A. Clay Stephenson · Accepted solution]

Look at maxdsiz, maxssiz, and maxtsiz. Shmmax can also come into play. Also, you might be limited by the amount of virtual memory thus you need to look at swapmem_on and the total physical memory + swap on each box.

Finally, even if all the settings are identical you might still have problems (or even invocations of the same program on the same box) because of memory fragmentation. There may simply not be a single contigious chunk of memory available to grant a large memory allocation request at a given moment in time.

[James Murtagh]

Hi Daniel,

First check if it is indeed a shmem_magic executable. You can run odump on it and cross reference that to the magic header file or see "man magic", odump will be in /usr/contrib/bin in case this is not in your path.

It will be either shmmax or maxdsiz depending on the above. If it is indeed shmem_magic you should raise shmmax. I would also ensure maxsiz is set as per the working systems.

Cheers,

James.

[James R. Ferguson]

HI Daniel:

You need to look at 'maxdsiz', available swap space (with 'swapinfo -tam') and 'smhmax'. Fragmentation of memory and the memory segment size can also account for the inability to find space for shared memory segments.
Regards!

...JRF...

[A. Clay Stephenson]

One more thing to check. Ulimit -a and look at the data, stack, and memory values. They might be set below the kernel allowed values.

[Daniel Gowans]

I went through and increased several kernel parameters. It turns out that the maxdsiz was set to 2 GB, but maxdsix_64bit was set to 1 GB! I am wondering if the 64bit size will limit a 32 bit app... I mean the 64 bit limit should always be larger anyway, so I don't know why it was set like that.

I also increased nproc, nfile, nflocks, and some other related parameters, as we were having file opening issues as well.

Thanks for all your help!

[James R. Ferguson]

Hi (again):

The kernel parameters suffixed with '_64bit' (e.g. 'maxdsiz_64bit', 'maxssiz_64bit' and 'maxtsiz_64bit') apply *specifically to 64-bit processes. Those not suffixed apply only to 32-bit processes.

Regards!

...JRF...

[James Murtagh]

Hi all,

After Daniel and JRF's last comments I remembered a call I dealt with in the long and distant past which had the same maxdsiz/maxdsiz_64bit characteristics, i.e. maxdsiz being larger than maxdiz_64bit and enomem errors. I couldn't quite remember quite what had happened so I did the following:

1) Set my maxdsiz to 1 GB and maxdsiz_64bit to 600 BM
2) Created 3 programs (attached):
a) A 64 binary that simply vforks and execs the others
b) A 32 bit binary that tries to malloc 800 MB
c) A 32 bit binary that tries to malloc 590 MB


As you will see the malloc on 800 MB fails even though maxdsiz is set to a Gig. I still don't know what exactly is happening (When I get back to work I WILL let you know) but it appears when a 64 bit executable forks a 32 bit process it assumes maxdsiz_64bit is larger than maxdsiz hence sets the limit to the former (assuming other dependencies are met I'd assume).

# ./vf
malloc error using 800 MB!
successfully malloc'ed 590 MB

Needless to say when I revert the kernel all is well with both malloc requests. You will note no setrlimit calls were made, I haven't tested the effect of this.

James, if you agress with the analysis i'll raise a request for this to be documented when I get back to work.

Cheers,

James.

[Bill Hassell]

Are the memory erors truly shared memory allocation errors or are they local data (malloc system call) errors? Without this information, it will not be easy to fix the associated kernel parameter(s). maxdsiz is only for local data. In order for a 32bit executable to malloc more than 900 megs, it must enable quadrant 1 (compiler or linker options: -Wl -N). You can determine this with file:

file a.out
a.out: PA-RISC1.1 shared executable dynamically linked -not stripped

--Q1+Q2 executable:

file a.out
a.out: PA-RISC1.1 executable dynamically linked -not stripped

The 'shared' option is removed from the Q1+Q2 executable. chatr will show the same difference.

So, if the executable is 32bits and having problems with malloc, then just maxdsiz will control the maximum size (set it to 2Gb).

[Don Morris_1]

Regarding the earlier mention of 64-bit parent exec'ing 32-bit children and their recieving the (lower) 64-bit limits.

Yes, that can happen. The reason is that children are supposed to inherit the limits of their parent process. Normally, if you're going 32/32 or 64/64 and no one changed a limit, that just means everyone gets the tunables by default. But when you go 32/64 or 64/32 then the kernel has to determine the proper values - and the algorithm in 11i uses the most restrictive values to be safe. (For 11i v1.5 and higher things are more complex due to these tunables being dynamic -- so the algorithm was revised to use the system-wide (tunable) values unless the parent explicitly modified the limit). The obvious workaround is to keep the 64-bit limit higher than the 32-bit limit.

There is not a patch in plan to address this issue.