mmap64 return ENOMEM(Err#12)

[Herbert Simon]

Hello Experts outside in the world.

While start a program I get an ENOMEM calling mmap64 ( traced with tusc ). Previous mmap calls on startup ( 32bit) runs fine. This issue rise only if one user open several programs which need lot of mem.
No system wide limit reached, as far as I can see.Same behavior using different users.

Q:
Why a mmap() 32Bit works but mmap64() not?

System data:
HP-UX slatibartfas B.11.11 U 9000/785 (C8000)
8GB Physical Mem

Swapinfo:
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 12288 0 12288 0% 0 - 1 /dev/vg00/lvol2
dev 3008 0 3008 0% 0 - 1 /dev/vg00/lvol36
reserve - 422 -422
total 15296 422 14874 3% - 0 -

Kmtune:
maxdsiz 0xc0000000 - 0XC0000000
maxdsiz_64bit 0x800000000 - 0X800000000
maxssiz 0x17f00000 - 0X17F00000
maxssiz_64bit 1073741824 - 1073741824
maxswapchunks 16384 - 16384
swchunk 2048 - 2048q

ulimit -aH
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 4293918720
stack(kbytes) 392192
memory(kbytes) unlimited
coredump(blocks) unlimited

ulimit -aS
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 4293918720
stack(kbytes) 392192
memory(kbytes) unlimited
coredump(blocks) 4194303

[Dennis Handly]

I assume your application is still 32 bit?
How large is your file? Are you mapping it shared? If so, you can fragment the shared memory space and not be able to mmap your large files.

While you can call mmap64 to map a large offset from the start but you probably can't map large files, entirely, since quadrants are limited to 1 Gb.

[Venkatesh BL]

Did you typecast the offset with (off64_t)?. Please try. If it fails, please let us know the actual arguments of the mmap64() call you use.

[Herbert Simon]

Hey, thanks for your answers.

Well, I'm not the programmer of this tool so I'm not able to change anything ;-(. Yes, it's a 32bit application. Here are the last important lines from tusc:

open("/tmp/Mw_th18eb/ADFA0EF8CB.46474.gma_heap", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) E
RR#17 EEXIST
open("/tmp/Mw_th18eb/ADFA0EF8CB.46474.gma_heap", O_RDWR|O_LARGEFILE, 0600) = 5
chmod("/tmp/Mw_th18eb/ADFA0EF8CB.46474.gma_heap", 0600) ............. = 0
fcntl(5, F_SETFD, 0) ...................................................................... = 0
mmap64(NULL, 16777216, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FILE, 5, NULL) ................ ERR#12 ENOMEM
close(5) .................................................................................. = 0
open("/usr/lib/nls/msg/C/strerror.cat", O_RDONLY, 0177777) ................................ = 5
fstat(5, 0xa80f33a0) ...................................................................... = 0
fcntl(5, F_SETFD, 1) ...................................................................... = 0
lseek(5, 0, SEEK_SET) ..................................................................... = 0
read(5, "m s g c a t 0 1 \0\0\0f9\001\001".., 1212) ....................................... = 1212
lseek(5, 3212, SEEK_SET) .................................................................. = 3212
read(5, "N o t e n o u g h s p a c e ", 16) ........................................... = 16
close(5) .................................................................................. = 0


For that you can see that the requested ~16MB is not to much. There must be somewhere a system wide resource exhausted. But maxdsiz and maxdsiz_64 are well defined. A stack overflow should not raise related to a mmap()...

[Don Morris_1]

Being MAP_SHARED|MAP_FILE -- it isn't swap, and it isn't maxdsiz [since that only deals with Private Heap sizing] or maxssiz [only Stack].

With just the part of the trace you showed us, I'm speculating a bit -- but I agree with Dennis... your most likely issue here is that the 1.75Gb of 32-bit shared address space is too full and/or too fragmented for this 16Mb mapping. That may be because prior MAP_SHARED or SysV shmget operations by the application already ate all the shared space -- or it may just be because the "other programs which need lots of mem" are using the shared space (my bet would be on the other programs).

The simplest solution would be to use a 64-bit version of the program if available (or recompile as 64-bit if possible).

Something else that may work is to use Memory Windows for this application to give it a separate shared address space (current link to the white paper is: http://docs.hp.com/en/943/memwn1_4.pdf -- but that could change, of course).

Another solution, of course -- would be to get the *other* programs to 64-bit, in a different Window or [if they have configuration options] using less shared space.

[Dennis Handly]

>you can see that the requested ~16MB is not too much.

Why use mmap64(2) for that small amount?
Can you call mmap(2) if mmap64(2) fails?
You did say "previous mmap calls" work but they could be much smaller.

>Don: That may be because prior MAP_SHARED or SysV shmget operations by the application already ate all the shared space

You're saying that it is just random that mmap(2) works, or this is larger?

[Don Morris_1]

Not enough data.

The prior calls could have been:

1) Smaller than 16Mb (significantly) and hence either got the last of the Shared virtual address space by size or by fitting into the fragmentation holes.

2) MAP_PRIVATE instead of MAP_SHARED and hence come from a different area in the first place.

3) New process mappings of shared objects already mapped by an existing process [most likely to me] such as the usual shared libraries like libc and whatnot. The shared virtual address space for these would already be allocated/in-use -- this just adds another reference count to the virtual object.

4) Before whatever eats all of the shared address space and hence a race.

[Herbert Simon]

Hey dear experts!

Thanks a lot for spend so much time to this issue. Sorry not to provide "the whole" thing of data; Just let me investigate time to the SHARED MEM limit before provide lot of possible useless information.
Well, I'm not able to change anything of the source. So I also do not know why a mmap64 was used for 16MB..
The Program I traced named ProductView, A 3D Visualisation tool. The "other programs" are CATIA V5, VPM, Mozilla, Java applications...

Question to scope of MAP_SHARED segment.
A true 32Bit HP-UX 11i program can allocate up to 4Gb mem in four regions. The std. size of 1GB of these regions can be manipulated. So far so good. But I do not understand why a used shared mem segment of one process have affect the space of an other process.
If I allocate a shared mem segment of 1GB (std. layout) does this means that NONE other process can allocate memory to his (I thought private ) shared mem?
The result will be that a shared mem space is a global resource of max 1.75 GB for 32Bit programs which I cant believe...

I will update you about more investigations on our systems ( well, affected up to more than 6000 C8000 workstations :-( )

regards
Karsten

[Don Morris_1 · Accepted solution]

Believe it.

HP-UX on PA (always) and on IPF (by default, though there's more flexibility) works in a Shared Address Space (SAS) model. This is primarily because PA does not support virtual aliasing very well (i.e. it is very tricky to correctly manage multiple virtual addresses all mapping to the same physical page of RAM -- get it wrong such that you have conflicting cache lines, and the box HPMC's). IPF does support aliasing, hence the introduction of Mostly Private Address Space, true Copy-on-Write, etc. on later releases as IPF-only features. [There are still performance issues with too many aliases in managing all those translations -- I wouldn't encourage folks to go run everything MPAS, just selected apps/workloads as needed.]

As such -- Private data is truly private (you have some "sharing" via Copy-on-write -- but it is really Copy-on-Access to avoid this, so the first real use gets a new page and avoids the alias). Shared data is in a _globally_ shared area. 32-bit defaults to using the 3rd and 4th quadrants, with 1/4th of Q4 on 11.11 reserved for 32-bit memory mapped I/O. Hence, the 1.75Gb _global_ shared address space for 32-bit PA apps by default.

Memory windows provides some work around by using different Q3's and/or Q4's for a sub-set of applications. For what you have in mind to occur, you'd have to have every single process in a different memory window [which wouldn't work since the shared libraries wouldn't be mappable quickly]. Each process getting a unique q3 would be the next stage (with q4 staying Global) but would be a big limit on truly shared data.
Hence Memory Windows -- where some q3's are shared with sets of processes and q4 by default stays Global.

[Herbert Simon]

Hey Don, Hey Experts!

So I played a little bit to find out the usage of Q4 and Q3 using shminfo just before start the "crashing" application ( PView ). The Q4 is fragmented to 1160 pieces, only 12348KB marked as free from possible 784312 KB. These 576 free fragments have a size from 4KB up to 120KB.
Well, no space in Q4 to allocate 16000KB on a block.
I try to establish a memory Window for this application ( PView, 32Bit, SHARE_MAGIC ) but this don't work. Set max_mem_window to 50, start Pview with setmemwindow $ID -c [progname] and check the usage of Q2/Q3 without to overload the system by started all "others ( CATIAV5, VPM , JAVA ... ). Both segments displayed by shminfo are not used ( marked as 1GB FREE ).

O.K. I assume that the program try to load shared libraries to Q4 which has Free segments, but they don't fit the requested space. Normally then the shared libs should mapped to Q3. I know, if this happens within a memory window I will run in some difficulties concerning the scope of it to other programs.
I do not understand why a memory window do not help.

Nevertheless, I can't change the binary to SHMEM_MAGIC which may help here nor recompile it to 64Bit ( which helps ).

Thanks a lot to all for spend so much time to this issue. And additional thanks to let me think more about these ( quite wriggled ) memory models.

Best regards
Karsten

P.S. If someone interested to analyse the shminfo file... Feel free and let me know your lightly thoughs :-)

[Don Morris_1]

What does `chatr Pview` (substitute actual path as needed) say, out of curiosity? [If Pview is already compiled/set to Q3 private that would fit your stated results and the shminfo output (which shows a ton of free space in Q3, which for a non-SHLIB which doesn't use MAP_GLOBAL mmap() request [mmap or mmap64 doesn't matter] should be the _first_ place the process looks.] Q3 private would also mean that it wouldn't matter when you assigned it to a new window... the global Q4 would stay the global Q4.

The only other possibility I can think of right now is that the file is already mapped by a process you don't know about to the global Q3 in such a way that the 16Mb mapping doesn't work out.