/usr/lib/dld.sl Not Enough space - How to Address this Issue?

[Alzhy]

The environment is a very busy Build Environment --- Cobol/C/C++/Java compiles with several Oracle 9 (64-bit) instances as Database repositories for the compiles/builds, versioing, logs, etc.

We have been experiencing an increasingf frequency of this error. Some from Cobol and some from Oracle as follows:

/usr/lib/dld.sl Call to mmap() failed - TEXT /opt/cobol/lib/libcobrts.sl.2
/usr/lib/dld.sl Not enough space
/usr/lib/dld.sl Call to mmap() failed - TEXT /oracle/app/oracle/product/9.2.0.6/lib32/libclntsh.sl.9.0
/usr/lib/dld.sl Not enough space


Where could this possibly be fixed? Are there kernel parameters that we need to be looking at increasing or is this just simply a resourse issue? The environment is almost always at 100% CPU utilization but Memory/Swap are healthy as do IOs.

Any ideas/similar experiences?

[Steven E. Protter]

Shalom Nelson,

What is shmmax set to?

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=861278

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=837639

Seems like a memory issue.

SEP

[Arunvijai_4]

Hi Nelson,

It is a memory problem. Post your #swapinfo -tam and # vmstat information to analyse further.

-Arun

[A. Clay Stephenson]

I'll assume that you have plenty of swap space. Your problem is the common shared memory address space for all 32-bit processes has become so fragmented that a request for a large chunk of memory can't be satisfied. Your problems will temporarily disappear if you reboot. The real solution to your problem is to convert everything to 64-bit code; the Plan B solution is to use memory windows so that each group of related 32-bit processes gets its own address space.

[TwoProc]

If the problem is not in the shared memory area (re: SEP)...

Since it says:
/usr/lib/dld.sl Call to mmap() failed - TEXT

It could be a failure for a memory allocation in a text segment. Is maxtsiz or maxtsiz_64bit too low?

[Alzhy]

Swap is ample - maximum util at 15% , Glance/Measureware history show only up to 99% memory utilization.

System is a 8-32 cpu vPar, 8GB of memory.

Is it possible this is due to short term maxing out of memory (RAM)?


max_mem_window 0 - 0
max_thread_proc 3072 - 3072
maxdsiz 0x80000000 - 0X80000000
maxdsiz_64bit 0x40000000 - 0X40000000
maxfiles 10000 - 1024
maxfiles_lim 10000 Y 10000
maxssiz 0x17000000 - 0X17000000
maxssiz_64bit 0x17000000 - 0X17000000
maxswapchunks 16384 - 16384
maxtsiz 0x18000000 Y 0X18000000
maxtsiz_64bit 0x18000000 Y 0X18000000
maxuprc 3072 Y 3072
maxusers 800 - 800
msgmax 65535 Y 65535
sema 1 - 1
semaem 32767 - 32767
semmap 4098 - (SEMMNI+2)
semmni 4096 - 4096
semmns 32767 - 32767
semmnu 16384 - 16384
semmsl 2048 Y 2048
semume 512 - 512
semvmx 32767 - 32767
shmem 1 - 1
shmmax 0x600000000 Y 0X600000000
shmmni 2048 - 2048
shmseg 2048 Y 2048

[A. Clay Stephenson]

No, physical memory has nothing to do with this; it's virtual memory that we are concerned about. I will say that your maxssiz (368MB) is way too large but that will limit data allocation (e.g malloc's) because the heap and stack share a common quadrant. By making maxssiz soo large, you limit the data segment even if the actual stack never approaches maxssiz. You should reduce maxssiz to no more than 64MB (and that is very generous) and that will instantly allow the data segment to be ~300MB larger. ---- NOTE: This still won't fix your mmap() problem but it's dumb to have so large a maxssiz value and only very bad code would need anything larger than about 64MB.

Shared objects and that includes shared-memory segments, memory-mapped files, and shared library objects are allocated from the same address space (so maxtsiz will not help); if you must remain in 32-bit land then memory windows are your answer.

[Victor Fridyev]

Hi,
As it was mentioned above, the problem is in free memory lack for new Oracle processes.
I've resolved such problems by increasing "shmmni" parameter in the kernel or decreasing the parameter "processes" in $ORACLE_HOME/dbs/init$ORACLE_SID.ora file.

HTH