- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- pthread_create()
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-08-2005 01:33 PM
11-08-2005 01:33 PM
pthread_create()
We are intermittently getting a "pthread_create() failed" error from a standard vendor supplied application in one of our environments alone. We are not able to reproduce this error though. The same application runs fine in other environments without any issue. All the environments are configured alike.
We have the following kernel parms
max_thread_proc = 1024
nkthread = 20000
nproc = 1461
We have monitored the system and have not seen any significant increase in pthreads. The process thread list count monitored via gpm does not go above 70 for this application. The memory utilization is not high as well.
What could be the probable cause for failure.
Detailed error message is given below.
(cthreadpool.cxx:424): cThreadPool_::Add(): pthread_create failed. (from OS) Resource temporarily unavailable (0x0000000B)
(cthreadpool.cxx:461): CreateThreadFailed E:0x20200000 (not open)
(svcmain.cxx:487): ServiceMain(): pThreads->Add() Failed
Thanks
Krishnan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-08-2005 02:31 PM
11-08-2005 02:31 PM
Re: pthread_create()
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-08-2005 08:19 PM
11-08-2005 08:19 PM
Re: pthread_create()
thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 04:54 AM
11-09-2005 04:54 AM
Re: pthread_create()
The HPUX version is given below.
HP-UX hp_prod B.11.11 U 9000/800 651359373 unlimited-user license
Our unix admins have verified the kernel parms on both systems and claim they are similar.
The kernel parms that differ between the 2 systems are given below.
Parm Bad Good
bufpages 125000 0
dbc_max_pct 0 4
dbc_min_pct 0 4
ksi_alloc_max 11688 22352
maxdsiz_64bit 8808038400 4294967296
maxfiles 2048 4096
maxfiles_lim 2048 4096
maxuprc 1168 2235
maxusers 400 800
msgmni 1461 2794
nclist 6500 12900
ncsize 18856 29520
nfile 23963 757760
nflocks 1461 2794
ninode 13736 24400
nproc 1461 2794
nstrtel 400 800
nsysmap 2922 5588
nsysmap64 2922 5588
semmap 2924 5590
semmni 2922 5588
semmnu 1457 2790
swchunk 2048 65536
unlockable_mem 4000 8000
The bad system is our production server.
Please let me know if changing any of the kernel parms above could help prevent this problem.
Thanks
Krishnan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 06:03 AM
11-09-2005 06:03 AM
Re: pthread_create()
BTW, from the pthread_create() manpage...
[EAGAIN]
The necessary resources to create another thread are not available, or the number of threads in the calling process already equals PTHREAD_THREADS_MAX.
And this happens to be the only error which can happen intermittently, all others are related to invalid args (though that too can happen if you keep changing args).
When this problem does occur, does it occur for multiple times in a bunch ?
BTW, the manpage also has a warning saying that if the threads are joinable but have not been joined, then whether or not they contribute towards PTHREAD_THREAD_MAX is undefined, but it may mean that they do, and "gpm" may not show them (not sure)!
- bd
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 06:17 AM
11-09-2005 06:17 AM
Re: pthread_create()
We have closely monitored the system and don't believe the PTHREAD_THREADS_MAX is being reached. This is currently set to 1024.
Since the log does not provide any more details how can we determine what other resources could be maxed out?
I also noticed that warning about joinable threads and was not sure if that contributes towards PTHREAD_THREADS_MAX. If so is there any way to tell if this value is being exceeded.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 07:42 AM
11-09-2005 07:42 AM
Re: pthread_create()
# kmtune -q max_thread_proc
thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 08:11 AM
11-09-2005 08:11 AM
Re: pthread_create()
nkthread 20000
nproc 1461
maxuprc 1168
Currently we have the above values on the production server which is the bad system. The good system also has the same values and we don't get this problem there.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 09:44 AM
11-09-2005 09:44 AM
Re: pthread_create()
maxdsiz
maxtsiz
maxssiz
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 10:22 AM
11-09-2005 10:22 AM
Re: pthread_create()
Again, EAGAIN is not a completely unexpected event during pthread_create and the software should cope with what is normally a transient situation --- that is the real fix.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 11:46 AM
11-09-2005 11:46 AM
Re: pthread_create()
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 02:37 PM
11-09-2005 02:37 PM
Re: pthread_create()
If you dont have glance and measureware on the system you can load it once and use it for 60 days to "evaluate" the product.
Good luck..
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2005 03:56 PM
11-09-2005 03:56 PM
Re: pthread_create()
Could you plz answer one of the questions in my previous post, i.e.--
when the pthread_create() error does happen, then do you see a bunch of these (&/ other errors), or this is the single error for a sometime. also, what is the frequency of this error ?
thanks,
banibrata
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-10-2005 08:09 AM
11-10-2005 08:09 AM
Re: pthread_create()
The pthread_create() error is noticed in the applications log. There are no associated problems seen either at the OS or with any other application. As I had indicated this is quite intermittent and we have not been able to reproduce this. The frequency is not consistent as well. Couple of times per week.
Our unix admins have been monitoring the systems closely with gpm, glance etc. and are not able to find any resource issues that might trigger this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-10-2005 01:01 PM
11-10-2005 01:01 PM
Re: pthread_create()
IMHO...the intermittency and frequency of the problem might correlate to load on the server. Your production server maybe getting impacted heavily during those two or three times/week spawning processes that create threads but don't destroy them.
As for your other system where this application runs fine, is that a development box? If that's the case then try ramping up the load on your development box so that it matches production and see if you come across this error again. I'm saying so because many a times the development systems are not stress-tested to production's capacity.
BTW, did you try increasing the max_thread_proc parameter? And what about the kernel values for maxtsiz/maxdsiz/maxssiz on both the systems.
cheers!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-10-2005 07:43 PM
11-10-2005 07:43 PM
Re: pthread_create()
1) max_thread_proc is a per-process limit, not a cumulative system limit.
2) nkthread OTOH is a cumulative system limit. so if you have other processes which momentarily create too many thread, of too many processes get spawned with few threads, the nkthread may be hit, even though for your application max_thread_proc is not hit.
so you need to watch out for no. of threads in your application, total number of threads in the system, while monitoring.
One thing you can do though is to reduce the max_thread_proc to say about 80 (since you said your application, at max runs about 70 odd threads), and then see if the frequency of error increases. This may help in faster and controllable reproduction of the problem.
If you are able to reproduce your problem more often, debugging shall become easier. I'd recommend this as the first step, not increasing any of the limits.
my 2 cents,
bd
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-17-2005 11:56 PM
11-17-2005 11:56 PM
Re: pthread_create()
Tru to install "tusc" to trace system calls, and check the mmap()/sbrk() calls for failures.