big time memory leak with java.exe on CIM7 SP2

[m cowperthwaite]

Hello,

I've got a dedicated Insight Mgr 7 sp2 550 workstation running win2k SP3 and SQL 2K with SP3. The wkstation is well populated with dual processors and 512 mem. I've noticed a real loss in perfomance and after checking I found that the java.exe app is or appears to be leaking memory. this causes the server to really slow up and the only way so far I've found to clear it is a reboot, but that does'nt work for long. Am I able to upgrade the Java to a latest version? or is there an other workaround or fix available?

[Jeff Mathews]

Hi,

I haven't heard of this problem before, but then again my IM server has 4 Gigs of RAM. How much memory is java.exe using? On my server it states about 60 MB. Sqlserv.exe is by far the biggest hog, using around 500 MB.

Jeff

[mark cowperthwaite]

hi,

This could be a case of a real need for more memory, the java.exe is running at the moment with 64Mb and the sqlserv is, to be expected at 343Mb. The java.exe is rising in it's use of memory, it generally keeps going up until one of us stops the IM7 service and restarts it. I've checked out SUN for updates of the java app and have downloaded the latest ver 1-4-1-02, however I don't know if the java ver for IM7 can be upgraded.

[Tim Pearce]

I too have this problem. IM7SP2 running on ML330G2, 256MB, MSDE2k, only around 50 Proliants monitored so I didn't think I'd need much more, yet. It runs fine most of the time, but every now and again, java.exe spirals out of control, and eventually exhausts the page file and the server needs a reboot. If you can restart the IM service before it gets too far then it recovers for a while.

I'm gonna try a little more RAM and see if it improves.

[David Claypool]

Unfortunately, certain Java classes in use are tied to a particular JRE, namely the 1.3.1_02 that is part of the IM7 SP2 distribution. One thing you might try to avoid reboots is the Workload Management Pack. You can restrict the amount of virtual memory a process has access to and restart the process when a limit is reached. Free download and 30-day eval license at http://h18013.www1.hp.com/products/servers/proliantessentials/rpmdescription.html

[Gaetano Giunta]

Maybe not a brilliant solution, but a quick one:
schedule a nightly script to stop and start the IM service?

e.g

net stop "Insight Manager 7"
net start "Insight Manager 7"

Anyway, it would be great if IM could be compiled with newer java VM, or AT THE VERY LEAST allow for them on the client side (why is 1.3.1_02 ok and 1.3.1_03 to _07 not???)

[James Kennedy_4]

Ugh! Yah, talk about annoying. Insight crashes the server its on about once every week or two.

And yeah, it is really annoying that the client side
can't have a newer version of Java installed.

Why doesn't IM work with the Microsoft VM. I thought the whole point of CIM7 was so you could use it from anywhere and any browser. Well you can, but nobody wants to put the Java client on their servers. They need to make it work with Microsoft VM as well.

[Darrin Rawls]

Typically, memory usage will grow when you start getting a SIGNIFICANT amount of traps/events. Events are held in memory for processing and additional manipulation. How many events are showing up in the ALL EVENTS query (not off the uncleared "table view")?

[Scott Shaffer]

Just in case anybody really wants to know...

As you can guess from recent court cases, the JVM from Microsoft isn't exactly compliant with Sun's standard. Sure, the basics are there, but as you can imagine Insight Manager 7 needs far more than just basic support. The main culprit is a UI library known as Swing (we use it for the list controls and other UI elements that make the UI easy to use) that doesn't run on Microsoft JVM.

Also, Sun has decided that they aren't interested in making their JVM's backwards compatible. That means that when we ship with 1.3.1_02 Sun doesn't have a problem breaking what functions we use in 1.3.1_07 or 1.4.1.

The bottom line is - you have to run what we built it or and tested with. For most folks this isn't a problem, but for people who need to run multiple applications that use Java, this can be a real headache.

And no, you can't update Java server-side either. Again we have the backward compatability problem.

As for the memory leaks - there aren't any known issues. If somebody would call it into customer support we can work it (unfortunately these types of things can't be resolved via a forum.)

[Sylvain Amyot]

Hi,

I have the same problem!

Just a comment to HP support, this forum is plastered with this issue from early versions of CIM to the latest, therefore I can't believe your statement about this being an unknown issue!

This is a bug that needs a fix!

[Scott Shaffer]

Let me clarify. I mean that this isn't a known issue with a fix in the works or workaround known. This is also not a systemic problem. We have 20+ Insight Manager 7 servers here in our QA labs and none of them are showing a memory leak.

I'm NOT saying that I haven't heard people say that they are seeing a memory leak in JAVA.EXE in IM7 (why leaks in older versions are relevant I can't see.)

If somebody (anybody!) with this problem would please call into support and open a case we can work it. As you can imagine, getting the details of your particular system and debug information on it is going to be too hard to do in a forum.

[Rob Buxton]

Scott,

I'm wondering if the "known issue" referred to the JRE Version issues rather than the Memory Leak issue that the original post was about?

I'll add I'm running IM 7 SP2 on a DL380 G2 with, admittedly 1.8GB of Memory, and I'm not seeing any problems where I need to reboot the Server.
IM is existing alongside a number of other applications that also use SQL Server databases on this machine.

[Brett Boone]

We at Reliant in Houston are expeirencing the identical problem that others are describing w/ java.exe climbing in memory consumption. A case has been opened w/ tech support and the engineering dept. has been working the case the last few months. They have not been able to duplicate the issue in their lab, but are making a special onsite trip to our shop to investigate futher tomorrow. I will let everyone know what we find if anything..

[Brett Boone]

Compaq engineers and I are 99.9% sure that we have found the root cause of the java memory leak. We believe that it is caused from servers that are/aren't in the database having incorrect snmp settings assigned in the trap and security tabs. With the incorrect settings and without 127.0.0.1 (local loopback adapter) set in the snmp properties, the server will send authentication failure traps to the console; which would flood IM7 and make java.exe rise with memory consumption. After running a debug tool such as network monitor to monitor the snmp traps coming in; we realized which servers were sending all the authentication failures as i described and fixed the snmp settings to the correct values.

All in all make sure that (ALL) servers in your environment have the correct SNMP settings. This would be the correct name/ip address of the IM7 server and the (read-only) community string for the trap tab. Also you will need the name/ip address and the local loopback adapter address (127.0.0.1) along w/ the read-only and read-write community strings on the security tab set correctly. Until these items were updated on each server in our environment, the memory leak still existed.

[Rob Buxton]

Brett,

Thanks for the update, certainly might save others some grief.