pthread_create fails with error 12

[rymani]

In our application(32-bit) I'm facing problem with pthread_create(), which says the error as 12. Application is running without any problem when the memory size is below 2004M (Monitored using top), once the memory reaches 2004M then the application is failing. So I looked into kernel parameter maxdsiz which was 0X80000000, I raised the maxdsiz to 0XC0000000 then also the application failed once it reaches 2004M.

I gone through this thread also "https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1158774".

These are some of the informations:

max_thread_proc = 1024
Machine information: "HP-UX B.11.11 U 9000/800"
aCC -V "aCC: HP ANSI C++ B3910B A.03.37"
what libpthread.sl
libpthread.sl:
Pthread Interfaces
$Revision: libpthread.1: @(#) depot-32pa CUP11.11_BL2001_1023_2 PATCH_11.11 PHCO_25226 Tue Oct 23 14:00:09 PDT 2001 $

Linker options: -D_REENTRANT -D_POSIX_PTHREAD_SEMANTICS -D_POSIX_C_SOURCE=199506L -mt -lpthread -lm

Suggestions Please.

[Luk Vandenbussche]

Your current pthread revision dates from 2001.

Please install a more recent version PHCO_36229

[Don Morris_1]

maxdsiz can't help you when you hit the architected quadrant boundaries.

top isn't a precise tool -- but I'm assuming your binary is set to EXEC_MAGIC (normal executable).. hence you have 2Gb of space for your private objects, and 2Gb of space for shared objects [keeping in mind that 11.11 PA reserves .25Gb of that global shared space for Memory Mapped I/O objects, so in practice 1.75Gb for non-MMIO shared objects].

If you want more than 2Gb of private space (which is where your pthreads are trying to go), you'll need to chatr your application with +q3p at least (and eventually go to q4p). Note that this trades off private for shared space -- and will mean that any Shared libraries you have must fit in the Fourth Quadrant, which is normally preferred for system libraries / libraries which are Global across Memory Windows. There may not be enough room in q4 for your libraries that will be moving from q3, so consider yourself warned.

The real solution, of course -- is to get out of the limitations of the 32-bit addressing model and recompile 64-bit. That's not often quick and cheap (though it can be), so I understand if you dislike the idea.

[rick jones]

And in some cases it may be necessary to compile the 32-bit application with (IIRC) -N or something to allow using some of those other spaces for data/heap/whatnot. (I'm networking not compilers, so my dimm memory will be slightly off on the syntax, but I think the gist of it is accurate)

Given that 64-bit has been possible with HP-UX for a decade now (11.0 FCS was November of 1997) it might not be a bad time to consider a 64-bit upgrade of your application(s)... if nothing else, there are options to the compiler (perhaps even one as old as yours :) to at least give 64-bit compatability warnings.

[Dennis Handly]

That link was for EAGAIN not ENOMEM.
I don't see you using -N as a linker option?
You must also compile with -mt.

>Rick: there are options to the compiler (perhaps even one as old as yours :) to at least give 64-bit compatibility warnings.

Probably not that old. +M2 needs A.03.52.
I suppose you could use this instead:
+Ww818,819,887,863,1036,1037,1038

Or better yet, download cadvise and use +w64bit:
http://www.hp.com/go/cadvise

[rymani]

Luk> Thanks for your valuable suggestion, we are looking into that.

Don> Yes, our application is EXEC_MAGIC. We tried with the +q3p enable option and we could see the difference, in terms of memory usage. Previously the application failed at 2004M and now it is not failing but it is hanging at 2098M, where the CPU% is showing as 99.9%. Is this option(+q3p) is having any relevance with CPU utilization?.
Our application is a multiplatform tool, so it requires a architectural change in order to go for 64 bit.

Rick> We are linking with -N option. What would be the difference if we add this as the compiler option?

Dennis> I'm sorry. I forgot to mention about the -N option in the linker options which i already mentioned. Below is some information.

CPPFLAGS = -D_REENTRANT -D_POSIX_PTHREAD_SEMANTICS -D_HPUX_SOURCE -D_HP_UNIX_ -D_HP_UNIX_ACC_ -D_POSIX_C_SOURCE=199506L

CXXFLAGS = -mt -AA +Z +eh +DAportable -w -Wc,-ansi_for_scope,off

MAKEDEPEND_FLAGS = +M

LDFLAGS = -Wl,+s -b -mt

LINKOPTS = -AA -Wl,-N -Wl,+s +eh -lpthread -lm

[Don Morris_1]

No, +q3p should have no effect on cpu activity. It only tells VM (that's Virtual Memory not Java Virtual Machine, mind you) to put Q3 in a private space and allow private objects there. From an application point of view, the only difference is higher addresses from MAP_PRIVATE mmap(), malloc(), etc. From a kernel point of view, this is just a matter of the space portion of the Global Virtual Address, the address map (or lack thereof) used for new allocations, etc.

This is pure speculation - but I'm suspicious that you crossed the 2048Mb boundary for a private pointer (i.e. pointer from malloc() / mmap(), etc.) and started looping. That makes me think there's a cast (implicit or explicit) of a pointer to a signed 32-bit integer in there somewhere... causing your private address to get truncated. Then you could be trying to step through what's now a negative loop state, etc. Running cadvise from Dennis's link is probably a good idea in general.

Have you tried attaching a debugger to the application to see where the presumed looping activity is?

[rymani]

Don> I'm yet to try cadvise suggested by Dennis, i attached the process to wdb and trying to find the loop.

Below is the information of the application taken using chatr.
normal executable
shared library dynamic path search:
SHLIB_PATH enabled first
embedded path disabled second Not Defined
shared library list:
.
.
. (our application libraries)
.
.
dynamic /usr/lib/libpthread.1
dynamic /usr/lib/libm.2
dynamic /usr/lib/libstd_v2.2
dynamic /usr/lib/libCsup_v2.2
dynamic /usr/lib/libcl.2
dynamic /usr/lib/libc.2
static /usr/lib/libdld.2
shared library binding:
deferred
global hash table disabled
plabel caching disabled
global hash array size:1103
global hash array nbuckets:3
shared vtable support disabled
static branch prediction disabled
executable from stack: D (default)
kernel assisted branch prediction enabled
lazy swap allocation disabled
text segment locking disabled
data segment locking disabled
third quadrant private data space disabled
fourth quadrant private data space disabled
data page size: D (default)
instruction page size: D (default)

[Dennis Handly]

>Don: That makes me think there's a cast (implicit or explicit) of a pointer to a signed 32-bit integer in there somewhere... causing your private address to get truncated.

That won't happen in 32 bit mode. But a signed vs unsigned compare would go wrong.

>Don: Have you tried attaching a debugger to the application to see where the presumed looping activity is?

Right, that's a good idea. You can also try attaching tusc to see if any syscalls are being done.

>i attached the process to wdb and trying to find the loop.

Did you find anything? You could just hit control-C every so often and get stack traces. Or use the "finish" command to see if you return. Then do that again for each frame.

>Below is the information of the application taken using chatr. normal executable

I guess that's EXEC_MAGIC.

>CXXFLAGS = -mt -AA +Z +eh ... -w -Wc,-ansi_for_scope,off

Since +eh is the default, remove it.
Also, using -w isn't a good idea. Better to use +W### to suppress specific warnings.
Any reason you need
-ansi_for_scope,off? You should really fix your code.

>LINKOPTS = -AA -Wl,-N -Wl,+s +eh -lpthread -lm

You don't need -lm or +eh. Also -N is a driver option so you don't need "-Wl,".