Operating System - Linux
1753656 Members
5728 Online
108798 Solutions
New Discussion юеВ

Re: OutOfMemoryError : unable to create new native thread

 
Varun_5
Occasional Advisor

OutOfMemoryError : unable to create new native thread

Hi,
I am using SunOne Application Server, and i am executing some test cases testing failover of session ( with server resatrt ) with entity Bean, HomeHandled reference stored in the session.
While executing my testcases started failing.
When i saw into the server log i found that it failing with following error complaining java.lang.OutOfMemoryError: unable to create new native thread.

Following is my system configuration :
Operating System : HP-UX B11.11
Architecture : PA_RISC
No. Of Processor : 4
Total Physical Memory : 4,192,256 kbyte
Free Physical Memory : 1,752,436 kbyte
Total Swap Space : 4,194,304 kbyte
Free Swap Space : 4,194,304 kbyte
Java Version : 1.5.0_01
I have Out Of Box installed on my machine.
Some pf the kernel parameter of Machine is :
max_thread_proc = 3000
maxdsiz = 2063835136
maxdsiz_64bit = 1073741824
maxfiles = 2048
maxfiles_lim = 2048
maxssiz = 8388608
maxssiz_64bit = 8388608
maxswapchunk = 2048
maxtsiz 67108864
maxtsiz_64bit = 1073741824
maxurpc = 75
maxuser = 512
maxvgs = 10
nproc = 2048
and ulimit is unlimited.

Error seen in server.log
==========================
[#|2006-04-19T16:49:50.994+0530|WARNING|sun-appserver-ee8.1|javax.enterprise.sys
tem.stream.err|_ThreadID=352;|java.lang.OutOfMemoryError: unable to create new n
ative thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:574)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.j
ava:833)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SS
LSocketImpl.java:1025)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.
java:619)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.ja
va:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65
)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198
)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:94)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl_Stub.getDefaultDomain(Unknown Source)
at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.c
heckConnection(RMIConnector.java:1543)
at com.sun.jmx.remote.internal.ClientCommunicatorAdmin$Checker.run(Clien
tCommunicatorAdmin.java:165)
at java.lang.Thread.run(Thread.java:595)
========================================
Please help me to understand the issue.
I cant see any problem with system configuration also.

Thanks,
Varun
8 REPLIES 8
Peter Godron
Honored Contributor

Re: OutOfMemoryError : unable to create new native thread

Varun,
is your maxdsiz_64bit correct?
Less than maxdsiz ?
Varun_5
Occasional Advisor

Re: OutOfMemoryError : unable to create new native thread

yes peter,

Value of maxdsiz_64bit on my machine is 1073741824
maxdsiz = 2063835136
maxdsiz_64bit = 1073741824
Peter Godron
Honored Contributor

Re: OutOfMemoryError : unable to create new native thread

Varun,
sorry that I did not make myself clear.
I think your maxdsiz_64bit is actually too low and should at least the same as maxdsiz.

I also found:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=211942 which seems to point in the same direction.
Varun_5
Occasional Advisor

Re: OutOfMemoryError : unable to create new native thread

Peter,
I have gone through that mail thread but it doesn't look like solving the stated problem. Is there any chance that test application or application server can be suspected ?
Sameer_Nirmal
Honored Contributor

Re: OutOfMemoryError : unable to create new native thread

Hi Varun,

Does the test case failure creates "core" file or "hs_err*.log" files?

Whats the patch level of the system?
Are the required patches for the Java version applied?

If the "core" file is being generated, you can use WDB or GDB to examine it.

You try reducing the thread stack size using "-Xss" option or increasing the heap size.

You might want to use "HPjconfig" tool to verify the kernel tunnables.

Refer the following link for details
http://www.hp.com/products1/unix/java/?jumpid=go/java
Varun_5
Occasional Advisor

Re: OutOfMemoryError : unable to create new native thread

Hi Sameer,
I will use HPjconfig and try to figure out if some more kernel parameter can be tunned for jvm.
I will keep posting about the progress.

Thanks,
Varun
Seth Kaplan
Advisor

Re: OutOfMemoryError : unable to create new native thread

Did you ever get this resolved? We're getting a similar error using JBOSS.

Seth
Bill Hassell
Honored Contributor

Re: OutOfMemoryError : unable to create new native thread

Just to clarify:

> maxdsiz = 2063835136
> maxdsiz_64bit = 1073741824

is quite wrong! If maxdsiz_64 is less than maxdsiz, then both parameters are silently enforced to the maxdsiz_64 value. ALWAYS set maxdsiz_64 larger than maxdsiz.

As far as OutOfMemory, there are many, many causes but generally it has nothing to do with physical memory. HP-UX is a virtual memory system so available memory is a combination of RAM plus swap space. But there are several limits that are much less than virtual memory. Kernel parameter maxdsiz sets the maximum local data (heap) for a 32 bit process. Parameter maxdsiz_64 refers only to 64 bit processes. shmmax refers to the largest shared memory segment allowed. If a set of programs need a larger segment, this kernel parameter must be adjusted.

To see all the ulimit values, use ulimit -a. If this fails, your shell is badly out of date (ksh needs a patch). Note also that shared memory (32 bit programs) must be mapped in a single space and can be heavily fragmented with InterProcess Comunication (IPC) objects like semaphores and other shared memory segments.

You may have a massively oversized kernel (normal is 8 to 15 megs) due to an oversized parameter, or your buffer cache is far too large (should be 600to 1200 megs).


Bill Hassell, sysadmin