Java Process consuming more of CPU

[kiran kumar kolluru]

Hi,

Setup: Alpha VMS 7.3-2.
Patches:
VMS_UPDATE V0300
VMS732_SYS V0600
VMS_FIBRE_SCSI_V0400
VMS_PTHREAD_V0200
VMS_TRACE_V0200
DNVOSIECO01_V0732
TCPIP_ECO_V5.4-154
System Main Memory is 2.00GB.
java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition
Fast VM (build 1.4.2-3, build J2SDK.v.1.4.2:08/20/2004-13:43,
native threads, jit_142)

From the recommendations from
http://h71000.www7.hp.com/ebusiness/optimizingsdkguide.pdf
for optimizing JAVA performance I have done the below steps.

UAF FILLM 4096
CHANNELCNT 4096
WSDEF 2048
WSQUOTA 4096
WSEXTENT AND WSMAX 16384
PGFLQUO 2097152
BYTLM 400000
BIOLM 150
DIOLM 150
TQELM 100

$ @SYS$COMMON:[000000.JAVA$142.COM]
JAVA$142_SETUP.COM;1 fast
$ DEFINE/SYS JAVA$CACHING_INTERVAL 1000
$ DEFINE/SYS/NOLOG
JAVA$DISABLE_MULTIDOT_DIRECTORY_STAT TRUE
$ DEFINE/SYS DECC$FILE_SHARING ENABLED

Still I did not see difference in usage of
CPU by JAVA process.

Did anybody let me know I missed any thing.

-Thanks in advance
Kiran

[Jan van den Ende]

Kiran,

you did not specify the hardware model, and, even more important, the amount of memory.

But, JAVA __IS__ a really CPU- and Memory-hungry beast!!

In your params I would like to increase WSEXTENT and BYTLM (at least) 4-fold, and I would remove the definition (in MODPARAMS) of WSMAX. After you do no longer specify WSMAX, let AUTOGEN calculate it for you (on the basis of physical memory).
After all, the memory HAS been payed for, why not let it be used on demand?

Hope this helps.

Proost.

Have one on me.

Jan

[kiran kumar kolluru]

Jan,

Thank you for reply.

Here the system specifications:

Model : Alpha Server DS10,CPU 600 MHz

Physical Memory Usage (pages):
Main Memory (2.00GB)
Total 262144
Free 109487
In Use 143672
Modified 8985
$1$DKA200:
Free Blocks:888030

I changed WSEXTENT and BYTLM to 4-fold and I did not have any definition for WSMAX in MODPARAMS.DAT

(I mean there is no value specified for WSMAX)
in SYSGEN the value of WSMAX is
SYSGEN> SHOW WSMAX
Parameter Name Current Default Min. Max. Unit Dynamic
-------------- ------- ------- ------- ------- ---- -------
WSMAX 157286 4096 1024 134217728 Pagelets
internal value 9831 256 64 8388608 Pages
-----------------------------------------------------------------

But I did not see much change in CPU utilization.

Please have a look at attachment for comparison.

-Thanks
Kiran

[Wim Van den Wyngaert]

What is this program doing ? Is it cpu bound ?

Is the process doing lots of pagefaults (sh proc/acc) ? If so, increase wsquota a lot more. It can be the reason of the slowdown.

It seems to me that the program gets what it asks and only consumes the cpu in a limited way (e.g. because of IO or network traffic).

Wim

[kiran kumar kolluru]

The process is Jcoreserver and it actually get the data with the devices attached using SCSI and java native calls.

Yes, we are observing a lot of pagefaults. I increased 'wsquo' ... no use.

-Kiran

[Wim Van den Wyngaert]

On our GS160, we have DESTA active which is also using Java. Its working set is about 200.000 pages (100 MB). So, as Jan said, Java is indeed eating a lot. I propose to try it directly with 500.000 pages and check if it improves.

Wim

[kiran kumar kolluru]

For information I am giving below the 'SYSTEM' quotas

Maxjobs: 0 Fillm: 4096 Bytlm: 1600000
Maxacctjobs: 0 Shrfillm: 0 Pbytlm: 0
Maxdetach: 0 BIOlm: 150 JTquota: 4096
Prclm: 25 DIOlm: 150 WSdef: 2048
Prio: 4 ASTlm: 250 WSquo: 500000
Queprio: 0 TQElm: 100 WSextent: 700000
CPU: (none) Enqlm: 2000 Pgflquo: 2097152

Actually I am using 'ECP Data Collector'
http://h71000.www7.hp.com/openvms/products/ecp/performance-and-capacity-download.html
for calculating the CPU usage.

With

$ plan collect =sample -
_$ /begin=21:05:00 -
_$ /end=21:15:00 -
_$ /interval=61 -
_$ /output=data.cpc

I am collecting the data and analyzing with

$Plan analyze/motif
Please have a look at attachment
for graph.
----------------------------------------------

With Java profiler

$ java -version
java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition
Fast VM (build 1.4.2-3, build J2SDK.v.1.4.2:08/20/2004-13:43, native threads, ji
t_142)

I am trying with "-Xrunhprof:cpu=samples,file=dka200:[kiran.pca]log.txt,depth=3"
which is showing 'jniOpen' and "jniScsi' calls taking more CPU. These calls
will call java native calls which will in-tern will fall into our application
code.

Hope this explanation will give you better idea about the environment.

-Kiran

[Jan van den Ende]

Kiran

"Jcoreserver"

I should have guessed!

I do not know WHAT that beast is doing (should just get some disk ststusses!!),
but the one who wrote it probably thought it a good idea to go to the other side of the street around the world!!

We are "lucky" enough to have dual CPU's in the systems, _AND_ the app is not multithreaded, but whenever the beast wakes up, one CPU (up to 1250 GHz) is 100% loaded by it for about half a minute.

--- just MAYBE, someone at Engeneering should look at that code with a pair of Performance-focused glasses :-(

Sorry.

Proost.

Have one on me.

Jan

[Willem Grooters]

Java is notorios bad in effeciency - both CPU and disk. One in a while (and you don't have ANY control over it!) the carbage collector will run. And since your Java process requires enormous amount of memory, it's likely to page memory to disk - and reverse to collect the carbage (not necessarily causing memory freed....).

The more memory a JAVA process will get, the better. But since JAVA is (in essence) an interpreter, there is a lot of CPU power needed; an extra porcessor (or more) is better.

Jan, I don't think Engineering can have a big hand in enhancing this. The environment must be checked (and approved by the (SUN-based) JAVA council and I do not believe that process-perfoamnce on other platforms than SUN matter much to them. THESE are the guys to be educated, not engineering)

Willem

[kiran kumar kolluru]

Hi,

I wonder why the 'Jcoreserver' doing something special on Alpha VMS.
Why it is -cool- at other OS !!!

-Kiran

[kiran kumar kolluru]

Hi all,

We have done a code change for some of calls made by java native libraries.

Some part of our C code is using stat() call to find the current size of log file.

We have replaced ftell() for stat().

Now I am seeing a good difference in utilization of CPU by 'jniScsi' and 'jniOpen'.

Why stat() is proving costly in Alpha-VMS ?

-Thanks
Kiran

[Ian Miller.]

what information are you acquiring by stat()?
Check the many CRTL feature logicals - there may besomething to help

[John Gillings]

Kiran,

> $ DEFINE/SYS JAVA$CACHING_INTERVAL 1000

and

>Why stat() is proving costly in Alpha-VMS ?

If you have Java code that calls stat() in a tight loop, the calls to stat() will generate lots of I/Os. (perhaps you're polling the size of the log file to see if it changes?)

The logical name JAVA$CACHING_INTERVAL causes "stat()" info to be cached, thereby reducing the I/O load, BUT it will cause a tight loop to become compute bound (since the stat results are returned immediately from the cache). It also means the repeated calls are pointless because the results are cached, so any changes out on disk will not be detected.

Your value of 1000 means 1000 SECONDS! So code looping on stat waiting for a change will loop for more than 16 MINUTES. That's a lot of wasted CPU.

The correct thing to do is put a delay in your stat() loop so that your program isn't polling the disk. This will reduce both I/O and CPU load. However, if you can't do that, you need to choose a value for the caching interval that balances I/O and CPU. Higher values will increase CPU, lower values will increase I/O. I'd guess a value of 30 would be much better.

I'd also recommend AGAINST defining this as a system wide logical name. Different programs will have different requirements for caching data.

[John Gillings]

Oh, and something else to think about when tuning Java applications...

There was once a time when increasing the memory available to an application was always a good thing for performance.

However, that doesn't always work for Java.

For best performance you need to set the maximum heap size the the *smallest* size that will correctly execute. Since Java doesn't have a "free memory" function, it allocates memory from the heap until it runs out, then performs a garbage collection to recover unused memory. If you set the heap size much larger than necessary, Java will simply leave more junk around. Overall paging is increased, since the application is touching more memory, and the garbage collections will take longer because there's more to be collected. By reducing the heap size, you (potentially) increase the *number* of garbage collections, but *decrease* their duration and keep the application in a smaller working set, so paging is decreased.

This is highly counter intuitive to those of us who grew up in a world where memory was expensive.

You need to carefully measure the peak heap size and set the maximum so that it's just higher than the high water mark.

[Jan van den Ende]

John,


You need to carefully measure the peak heap size and set the maximum so that it's just higher than the high water mark.


Sounds interestingly promising.

Forgive my lack of knowledge in this, but HOW do I do that, and HOW do I get the metrics to base my settings upon?

TIA

Proost.

Have one on me.

Jan