Java Based Process & Memory Management

[Amit Dixit_2]

Hi,
I hava a java process running on my HP Box.
There are few things which I want to know..

1. How much of memory can be allocated to a
java process does it has certain limitation
by JVM not possible beyond 2 GB ?

2. Is there any script which can monitor
that process 24 Hours for CPU & MEM
utilization

4. Best practices for optimizing JVM on HP-UX

I am currently running JVM latest version
and HP-UX B 11.11

Thanks,
Amit.

[Steven E. Protter]

Java is sort of an infrastructure piece and I don't know of a lot of tools for "optimizing" it.

What I've seen on my systems with java depends on optimizing the applications that use Java.

Most of the java use on my systems is by Oracle products,which utilize it extensively. I've been very frustrated by seeing all kinds of java processes running with huge amounts of memory and not knowing why.

The answer was in the Oracle database and application server configuration files. We optimized those according to Oracle guidelines and although use was high, it stopped bottlenecking the system.

We also ran into a few Oracle bugs that caused what looked like java problems but were just application server problems.

So, the answer is it kind of depends. As a reference, I'm attaching the imprecise tools I used to identify the bottlenecks java but really oracle was causing.

SEP

[Bill Hassell]

1. The Java code most likely allocates what it needs. If there is some sort of config file for the Java code that allows for specifying memory usage, then you can allocate as much as you want. Unless the documetation for the Java code spells out the steps needed for 32bit versus 64bit executbles, you'll be limited to less than 960megs for 32bit executables, or with EXEC_MAGIC, perhaps up to 1750megs. However, you'll need to recompile or relink the JVM code to use memory above 960megs.

2. There is no special script, just write a cron job to use sar and monitor usage. Note that HP-UX is a virtual memory system so you can run thousands of megs of code in just hundreds of megs of RAM. Of course, the processes will be paging excessively so performance will be very poor. In that case, you just add (a LOT more) memory. And for threaded Java code, you may need more CPUs.

4. There are special practices other than to avoid using inefficient Java routines in the design of the application.

[lawrenzo]

the answer to your third question is that you can create a cron job to run a script with various commands such as sar top etc but the best we use is measureware software, search the internet and HP site.

lawrenzo

[Amit Dixit_2]

Bill,
We do have a config paramter to allocate
memory to JVM and that is in the server
startup script.

We have set it to 2048 MB.

We have a built in monitor within the
tool which shows the total mem consued
out of that 2 Gigs which normally hobber
around 54%.

But as the day progresses memory usage
keeps on increasing and ultimately we
have to restart the application.

How to solve this problem.
Is it limitation with HP to use only
2048 MB for JVM config.

We do have oracle but that is running on
different box.

No problems with CPU usage.

ONLY MEM is the bottelneck.

Thanks,
Amit.

[Bill Hassell]

There are common Java routines that have had memory leaks in the past.That is typically what happens when processes continue to grow in size. Now it's important to know whether the process(es) are designed to grow in size. A memory leak is defined as a programming error where allocated memory is supposed to be returned to the system but is not, or memory for the same purpose is constantly re-requested, thus wasting space. These types of memory leaks are programming errors and somewhat common in Java coding.

Now, is the memory (2Gb) local memory to the Java code or is it shared memory that several processes are using? The 960meg rule for 32bit programs (probably what your Java code is running) is also a limit for shared memory. There is a shared magic feature that allows shared memory to grow to 1750megs *BUT* all programs must be recompiled or chatr'd to use the additional 1Gb quadrant. And additionally, there is only one shared memory map for 32bit programs which all processes must use. This map can become fragmented and a process may not be able to allocate a required segment even though there are many smaller chunks that add up to more than enough.

These limitations are all a function of 32bit programs--they all disappear when the applications are recompiled as 64bit programs. If you do not have the source code or the programmers to fix the code, about all you can do is to get all the latest patches and report the mmemory leak to the supplier. To monitor which processes are growing in size, use something like this in a cron job:

UNIX95= ps -eo vsz,pid,ruser,args | sort -rn | head -50 >> /var/tmp/jvm.log

HP-UX is a virtual memory system so even if you don't have enough RAM, processes can continue to grow up to the amount of swap space you have. However, processing performance will be terrible once continuous paging occurs. swapinfo -tm will show you if swap space is in use. Since you're running 11.11, the buffer cache will shrink down as processes need more RAM, but it is a good idea to reduce dbc_max_pct to a percentage that works out to 400-600megs max.