topic Re: Problem starting 3,5Gig Java Process in Operating System - HP-UX

Problem starting 3,5Gig Java Process

cartman_2 — Wed, 25 May 2011 09:45:43 GMT

Hi,

We are getting a Java out of memory error when starting a Java process with 3,5 gigs.

These are the errors in syslog:

May 23 13:48:10 or syslog: Java out of memory messages are marked with pid:21629 in /var/adm/syslog/syslog.log.
May 23 13:48:10 or syslog: pid: 21629 4Kb 4Kb 4Kb PCT START/ 4Kb DEV NO./
May 23 13:48:10 or syslog: pid: 21629 TYPE AVAIL USED FREE USED LIMIT RESERVE PRI FILE NAME
May 23 13:48:10 or syslog: pid: 21629 dev 1048576 0 1048576 0.00% 0 - 1 64 (major) 2 (minor)
May 23 13:48:10 or syslog: pid: 21629 reserve - 1466327 -1466327
May 23 13:48:10 or syslog: pid: 21629 memory 4177157 1068791 3108366 25.59%
May 23 13:48:10 or syslog: pid: 21629 Dynamic libraries:
May 23 13:48:10 or syslog: pid: 21629 /opt/java1.5.11/opt/java1.5/jre/bin/IA64N/java_q4p
May 23 13:48:10 or syslog: pid: 21629 text:0x00400000-0x00428130 data:0x00429000-0x0042b308
May 23 13:48:10 or syslog: pid: 21629 /usr/lib/hpux32/libpthread.so.1
May 23 13:48:10 or syslog: pid: 21629 text:0xf76cc000-0xf7768000 data:0xf76c9000-0xf76cb4e8
May 23 13:48:10 or syslog: pid: 21629 /usr/lib/hpux32/libdl.so.1
May 23 13:48:10 or syslog: pid: 21629 text:0xf76c5000-0xf76c8d70 data:0xf776d000-0xf776d330
May 23 13:48:10 or syslog: pid: 21629 /usr/lib/hpux32/libc.so.1
May 23 13:48:10 or syslog: pid: 21629 text:0xf7443000-0xf76c4ec0 data:0xf7431000-0xf7442128
May 23 13:48:10 or syslog: pid: 21629 /opt/java1.5.11/opt/java1.5/jre/lib/IA64N/server/libjvm.so
May 23 13:48:10 or syslog: pid: 21629 text:0xf5c8f000-0xf7427270 data:0xf5a2f000-0xf5c5ec90
May 23 13:48:10 or syslog: pid: 21629 /usr/lib/hpux32/libm.so.1
May 23 13:48:10 or syslog: pid: 21629 text:0xf5488000-0xf55d9260 data:0xf5c6c000-0xf5c6cc88
May 23 13:48:10 or syslog: pid: 21629 /usr/lib/hpux32/librt.so.1
...................

at the moment we try to run the process swap is looking like:

/var/adm/syslog> swapinfo -atm
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 0 4096 0% 0 - 1 /dev/vg00/lvol2
dev 2000 0 2000 0% 0 - 2 /dev/vg00/lvswap
dev 1024 0 1024 0% 0 - 2 /dev/vg00/lvswap2
dev 3008 0 3008 0% 0 - 1 /dev/vg00/lvswap3
reserve - 5377 -5377
memory 16317 4175 12142 26%
total 26445 9552 16893 36% - 0 -

and the kernel max/s:
/var/adm/syslog> kcusage
Tunable Usage / Setting
=============================================
dbc_max_pct 9 / 10
maxdsiz 75329536 / 3863835136
maxdsiz_64bit 33554432 / 8294967296
maxfiles_lim 477 / 4096
maxssiz 1179648 / 134217728
maxssiz_64bit 524288 / 1073741824
maxtsiz 520192 / 1073741824
maxtsiz_64bit 117440512 / 1073741824
maxuprc 24 / 4000
max_thread_proc 84 / 3000

So it doesn't look like we have swap reservation problems, nor hitting kernel max parameters.

I think they are running the app with:

/opt/java1.5.11/opt/java1.5/jre/bin/IA64N/java -server -verbose:gc -XX:+PrintGCDetails -Xms1024m -Xmx3500m ..etc

Could it be a 32bit quadrant problen, and the 64bit flag -d64 is needed ?

any ideas?

Thanks

Re: Problem starting 3,5Gig Java Process

Matti_Kurkela — Wed, 25 May 2011 13:30:07 GMT

> Could it be a 32bit quadrant problen, and the 64bit flag -d64 is needed ?

If my understanding is correct, that's what it most likely is.

The only way for a 32-bit program to have enough space for 3500 MB of data would be to run as EXEC_MAGIC with the +q4p chatr flag enabled. This gives you about 3.8 GB of address space for your program code (including all the libraries) and data, total.

The flipside is that you won't be able to use System V shared memory nor memory-mapped files. I'm not certain if the JVM could run in this mode.

I'd definitely recommend using the -d64 flag.

MK

Re: Problem starting 3,5Gig Java Process

cartman_2 — Wed, 25 May 2011 14:28:21 GMT

Hi, Matty

A couple of things I forgot to mention, I am running a itanium box, so I think I should use mpas(Mostly Private Address space) that would equal the +q4 options in pa-risc.

The thing is that i am running the java_q4p that already has mpas enabled:

rx3600:/opt/java1.5/bin/IA64N#chatr java_q4p | grep add

address space model: MPAS

And the other thing I forgot to mention is that I have another 32bit java process running on the server.

virtual physical swap
pid ppid pages / bytes pages / bytes pages / bytes command
9640 1 919251 3.5g 846693 3.2g 853918 3.3g java_q4p
14440 1 638386 2.4g 57037 222.8m 60295 235.5m java_q4p

So I understand that both have to share the 3'8 gigs between them, that makes the case even worse

Re: Problem starting 3,5Gig Java Process

likid0 — Thu, 26 May 2011 12:37:36 GMT

Yes the space is shared for all.

Here you have a great HP doc, that explains the problem quite well

wget http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02645664/c02645664.pdf

You also have a trouble shooting doc, not so good, but also gives you and idea:

wget http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02791255/c02791255.pdf

Re: Problem starting 3.5Gig Java Process

Dennis Handly — Sat, 30 Jul 2011 17:05:31 GMT

>Could it be a 32bit quadrant problem, and the flag -d64 is needed?

Yes as Matti says. By specifying 3500, the java driver will select the most appropriate java executable, for your case, MPAS.

>I understand that both have to share the 3.8 gigs between them, that makes the case even worse

No, there is no need to share, each is mostly private.

So unless you have 32 bit JNIs, use -d64.

>Daniel: Yes the space is shared for all.

Not for MPAS.