java.lang.OutOfMemoryError: unable to create new native thread

[James Murtagh]

Hi Vilas,

When you choose the patch from the ITRC it then gives you an option to "add to selected patch list". This screen then shows the patch plus its dependencies with a ticked box to the left. If you untick that box to de-select PHKL_18543. If you still have problems you can always swremove the patches and install them again. The standard install, without the reinstall=true option will note PHKL_18543 is already installed and skip it.

You can always go ahead and try and fix the immediate problem before this though, the patches in question are vxfs filesets and don't appear to be causing any issue with the kernel rebuild. Note one step I missed earlier, before the kmupdate if you copy the newly configured system file in /stand/build to the /stand directory.

Regards,

James.

[Vilas]

Once again Thanks James,

We finished installing patches PHKL_23193 and PHKL_24011. At present installing java related patches. Please check the attached output of the check_patches utility. It gives warning regarding swconfig. So, is this warning ok? It says,
WARNING: One or more superseded patch filesets are in the installed
state, but they include a configure script. Since the patch
fileset is not configured, it means that the configure script
was not successfully executed when the patch was installed.
Since these patch filesets have already been superseded on
this system, it may not be safe to have the configure script
executed via the swconfig(1M) command. Doing so could
have a negative effect on the active patch.

Thanks & Regards,

Vilas

[James Murtagh]

Hi Vilas,

I think the warning can be safely ignored. However, if you want to be extra safe you could move the configure scripts to another directory so swconfig would not attempt to configure them. If you remove the superseding patch you may have to copy the configure script back and run it but this is unlikely, I haven't seen any issues with this, its a very common warning.

Regards,

James.

[Vilas]

Hi James,

Our System Admin has problem downloading java patches. He is trying since last Monday, but eachtime when he try to untar the downloaded file, it says 'incomplete file..'. Do you have any idead about this problem?

Thanks & Regards,

Vilas

[Vilas]

James,

Our system admin downloaded the patches individually and loaded them.
But this bundle for the java patches contains some patches which are related to korean trutype font patch....xfont server patch....japanese trytype font patch...

So, our system admin is asking, 'are these patches really required?'.

Also, after applying all the patches and setting the required kernel parameters, I still get the same OutOfMemory error.


Thanks & Regards,

Vilas

[Vilas]

Hi James,

Please check the attached tusc.tar.gz file for the output of the tusc. On screen it gives following error: (Please check next message for kmtune.output, as it doesn't allow to attach more than one file.
# tusc -p -v -f -l -u startExamplesServer.sh > tusc.output
+ /usr4/apps/bea/jdk131_08/bin/java -Xms32m -Xmx200m -Dweblogic.security.SSL.tru
stedCAKeyStore=/usr4/apps/bea/weblogic7/server/lib/cacerts -Dweblogic.security.S
SL.ignoreHostnameVerification=true -classpath /usr4/apps/bea/jdk131_08/lib/tools
.jar:/usr4/apps/bea/weblogic7/server:/usr4/apps/bea/weblogic7/server/lib/weblogi
c_sp.jar:/usr4/apps/bea/weblogic7/server/lib/weblogic.jar:/usr4/apps/bea/weblogi
c7/server/lib/webservices.jar:/usr4/apps/bea/jdk131_08/lib/tools.jar:/usr4/apps/
bea/weblogic7/samples/server/eval/pointbase/lib/pbserver42ECF183.jar:/usr4/apps/
bea/weblogic7/samples/server/eval/pointbase/lib/pbclient42ECF183.jar:/usr4/apps/
bea/weblogic7/samples/server/stage/examples/clientclasses:/usr4/apps/bea/weblogi
c7/samples/server/stage/examples/serverclasses:/usr4/apps/bea/weblogic7/samples/
server/stage/common:/usr4/apps/bea/weblogic7/samples/server/stage/examples/clien
tclasses/utils_common.jar -Dweblogic.Name=examplesServer -Dbea.home=/usr4/apps/b
ea -Dweblogic.management.username=weblogic -Dweblogic.management.password=weblog
ic -Dweblogic.ProductionModeEnabled= -Djava.security.policy=/usr4/apps/bea/weblo
gic7/server/lib/weblogic.policy weblogic.Server
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start(Native Method)
at weblogic.kernel.ExecuteThreadManager.startThreads(ExecuteThreadManage
r.java:177)
at weblogic.kernel.ExecuteThreadManager.setThreadCount(ExecuteThreadMana
ger.java:158)
at weblogic.kernel.ExecuteThreadManager.(ExecuteThreadManager.java
:81)
at weblogic.kernel.Kernel.addExecuteQueue(Kernel.java:368)
at weblogic.kernel.Kernel.addExecuteQueue(Kernel.java:358)
at weblogic.jms.dispatcher.DispatcherImpl.(DispatcherImpl.java:63)
at weblogic.jms.dispatcher.DispatcherManager.(DispatcherManager.ja
va:57)
at weblogic.jms.dispatcher.DispatcherManager.create(DispatcherManager.ja
va:114)
at weblogic.jms.JMSService.initializeCommon(JMSService.java:377)
at weblogic.jms.JMSServiceServerLifeCycleImpl.initialize(JMSServiceServe
rLifeCycleImpl.java:54)
at weblogic.t3.srvr.ServerLifeCycleList.initialize(ServerLifeCycleList.j
ava:54)
at weblogic.t3.srvr.T3Srvr.initialize1(T3Srvr.java:782)
at weblogic.t3.srvr.T3Srvr.initialize(T3Srvr.java:594)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:282)
at weblogic.Server.main(Server.java:32)
( Terminating process 2270 ("/usr4/apps/bea/jdk131_08/bin/../bin/PA_RISC2.0/nati
ve_thread") )
tusc: ttrace(TT_PROC_EXIT, 2270, 0, 0, 0, 0): Permission denied
[1] + Killed . ./startExamplesServer.sh

Thanks & Regards,

Vilas

[Vilas]

And here is the output of kmtune.

[James Murtagh]

Hi Vilas,

No, those patches should not be required.

This is strange....you will see (after Paddy's comments) that Java is requesting a max of 200 MB of heap. Threads do share the data segment address range so it may be the maxdsiz is too small (also update maxdsiz_64bit to at least maxdiz or above if you change this). This is all I can think of right now...maybe change your maxdsiz to 940 MB and retry. Otherwise you will need to go the APP vendor I think.

Cheers,

James.

[Vilas]

Hi James,

I tried setting maxdsiz and maxdsiz_64bit to 940MB, but I get the same OutOfMemory error.

Thanks & Regards,

Vilas

[Tim Adamson_1]

Hi,

I am coming in on the end of this long thread so I apologise if I repeat any suggestions.

From what I can see, you only have 512Mb of swap and the system has about 1Gb of RAM. I think the settings for maxdsiz are probably high enough (I would probably set it at 1Gb) but I don't think you have enough swap.

If you only have 512Mb, then add another 1Gb and see if that helps. You may be running out of swap!

Tim

[James Murtagh]

I am almost at a loss, certainly on 11.00. If you were on 11i then maybe we could do some kernel tracing....in any case your last bet is to increase nkthread (double it). I almost guarantee this will not fix the problem but with native threads (kernel entitees) as opposed to green threads (thread scheduler entities) this will not fix the problem. You may need to contact the App vendor.

[Vilas]

Hi James,

I took help of BEA support and now our application is running smoothly without any errors.
But I have one question. Whatever we did and tried everything was ok; the problem was with kmtune. I tried setting kernel parameters using kmtune like,
# cd /stand/build
# cp ../system .
# mk_kernel -s system

I rebooted the system after it generated following output:
Generating module: krm...
Compiling conf.c...
Loading the kernel...
Generating kernel symbol table...


But I don't know what is the difference between various methods like SAM, kmtune and sysdef. Because after setting parameters using kmtune, when I used 'kmtune' to check the values, it did show me that the values are set correctly. But when I used 'sysdef', it showed different values. And with SAM, it was showing all the default values. SAM was not at all showing values which I set using kmtune. So, when I set the values through SAM, it set the values correctly and I could start weblogic server without any errors.

Once again thanks for your valuable help.

Regards,

Vilas

[James Murtagh]

Hi Vilas,

Its good that you've got it working at least. Before hpux 11.x administrators typically used sysdef to get the value of most kernel parameters. Starting at hpux 11.00 the kernel setup and configuration changed to support dynamic tunables and modules. Although at 11.00 this wasn't fully implemented, the new tools to manage the system files and linking of the kernel were put in place. You could still vi the system file but this was strongly discouraged as it could lead to formatting problems hence to kernel build failures. So now kmtune and kmsystem are used to modify or query the system parameters or drivers. Normally this work takes place in the /stand/build directory, hence the procedure is getting a current copy of the system file, either by copying it from /stand or using system_prep and editing this. So, to change maxdsiz in the system file in /stand/build, use :

# kmtune -s maxdsiz=268435456 -S system

Note the uppercase S when refering to the system file. If you leave this out the default system file is used, /stand/system. So it is possible you were reconfiguring the kernel using the old values, hence the discrepency. kmtune will also query the master files in /usr/conf/master.d and the system file when used without any options - however it should show active and planned values for the parameters. SAM is just a GUI for what has been discussed, its probably better to use this if you are unsure.

There are various documents that describe all this, online and man pages. See the "System Administration" section here for more info :

http://www.docs.hp.com/hpux/11.0/index.html

Cheers,

James.

[Vilas]

Thanks James,

Also, thanks everyone who helped on this issue.

Regards,

Vilas