JVM running lengthy Full GCs

[Pavel Moravec]

Hello all,
we are running a Java application that is sometimes being stopped in its execution. We investigated that it is caused by Full Garbage Collector being run immediately after normal GC. See attached GC log. Since our application is a throughput one, we can't allow such stop-the-worlds that delay from 7 to 10 seconds. Those delays are seen even if the application is put outside live traffic and is (almost) idle.

We are using parameters "-Xmx2048M -Xms2048M -Xmn682M -Xss128k" for the JVM. We used that parameters in past many times with no such behavior. Searching on web I found some recommendation to add parameters "-XX:ParallelGCThreads=5 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=20 -XX:MaxTenuringThreshold=10" that really helped a lot - no Full GC has been executed during one day test.

However I feel those parameters are only a workaround and there should be a reason for that unexpected behavior - as I wrote we have never seen that despite deployed the application on many sites.

Some technical details follow:

uname: HP-UX XXX B.11.11 U 9000/800 3248519855 unlimited-user license

java -version: build 1.5.0.13-_09_may_2008_11_35

dmesg | tail:
Memory Information:
physical page size = 4096 bytes, logical page size = 4096 bytes
Physical: 8386560 Kbytes, lockable: 6446736 Kbytes, available: 7406260 Kbytes


Any help is really appreciated.

[Steven E. Protter]

Shalom,

This type of behavior is seen from time to time.

http://www.hp.com/go/java

install recommended patches and more current versions of java

SEP

[Dennis Handly]

Have you talked to the Response Center?

Have you tried tools like jmeter?

[Wim Rombauts]

Many things have an impact on garbage colection behaviour. Sometimes a really small change in the application, or in the usage of the application, can push the JVM over the edge and make it start frequent long-running garbage collects.
Has this installation more users ?
Is there more data to handle ?
Are users using this application more intensively ?
Patching the JVM is certainly another issue to look at maybe even you are running a patched JVM here while your other installations are unpatched.
Patching on the OS-level may also have an impact.

[Pavel Moravec]

Thanks for the replies so far.

I tried profiling tools / parameters though either JVM was unable to start due to memory (too verbose information required) or insufficient info were obtained.

I am about to upgrade to the latest Java version.

Btw. there are 2 boxes running the same application. Boxes are identical (kernel, HP patches), Java is the only difference I found. With an older version 1.5.0.04 no problems were detected.. And with Java 1.5.0.13 the issue appears regardless a throughput - it also appears when the application has been idle from its start.

Anyway I will upgrade the Java and check if problems persist.

[Pavel Moravec]

So finally we found that it was caused by using RMI. This system was utilised by our source code and the Full GCs were caused by explicit callings of System.gc() method in the RMI. Using JVM option -XX:+DisableExplicitGC resolves the problem completely.