After Upgrade to 5.3 Insight manager service needs restart daily

[Russ Todd]

After upgrading my SIM from 5.2 to 5.3 I now have a issue where each day I need to restart the hp systems insight manager service because I cannot get to the system status pages. Each day I get the error "communication with HP SIM server has been lost"

I do not see any application or system events in the log to indicate any service has stopped, just the opposite really. nothing seems to have stopped runing , just not responding.

The task manager shows that the mxdomainmgr service is consuming about a 1GB ram - which constant through the day.

I have about 400 systems being monitored.

Any thoughts of where I could look for the issue?

Central management server:

Operating system: Windows 2003 R2 ENT 64 SP2
Version: Systems Insight Manager 5.3 - Windows
Build version: C.05.03.00.00
Build date: 2008-12-10 15:46
Patches:
HP BladeSystem Integrated Manager 3.5
HP Insight Power Manager 2.0
HP Virtual Machine Management Pack 3.6
Remote Support Configuration and Services 1.0

[Brian Hahne]

I bet you have the same issues I do, and

Look in your logs folder...
\\...\Program Files\HP\Systems Insight Manager\logs

look at mxdomainmgr.0.log

see if you have java out of memory errors.

seems to be a something people are getting

[Russ Todd]

I don't see anything specifc to memory errors. I did a straight search for "memory" to see if I could find anything that linked to Java out of memory.

I include the past 4 logs, maybe you know better what to look for...To confirm this is the problem.

Additional information: over the past couple of days I have only restarted the insight manager service, today I rebooted the entire machine.

Any additional ideas are appreciated.

[Torben Holm]

I´ve just upgraded to SIM 5.3. I also have to restart SIM to get it working again. The server consumes a lot of memory now.

[AdamJ]

II have the same problem. Has anyone managed to resolve this?

[SJO EGGER]

hello

same problem - call still open at HP. no idea at this moment.
did not update to 5.3 at this time would be the best.

just one question: which database did you have?

regards
roland

[James Kennedy_4]

I'm having the same problem as well. Brand new server build with fresh install of SIM 5.3. Attached to remote SQL 2005 database server.

[JKytsi]

And same here too ... SIM just dies.

updated from 5.2.2 to 5.3 oracle DB

mxdomainmgr service takes ~1,3GB memory
and 2500 systems to manage

I have enabled some debugging in SIM

INFO [HPSIM_DEBUG] [MapleConnection-5] WARNING: Result set not closed? for ID: updateHProductName Developers Please fix this

[JKytsi]

Is there any downgrade path to SIM 5.2.2 ?

ERROR [JDBCTransaction] Commit failed
java.sql.SQLException: IO-poikkeus: No buffer space available (maximum connections reached?): recv failed
25 helmi 09:38:24,504 INFO [WebappClassLoader] Illegal access: this web application instance has been stopped already. Could not load org/apache/axis/configuration/EngineConfigurationFactoryServlet.class. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
25 helmi 09:38:24,504 INFO [WebappClassLoader] Illegal access: this web application instance has been stopped already. Could not load org/apache/axis/configuration/EngineConfigurationFactoryDefault.class. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
25 helmi 09:38:25,692 INFO [TransactionManagerService] Stopping recovery manager
25 helmi 09:38:26,942 INFO [Server] Shutdown complete
Shutdown complete
Halting VM

[Brian Hahne]

That sure would be nice.
V5.3 (plus ILO hotfix) is useless if you need it to stay up more than 24 hours.

[Gus Tonsay]

SIM 5.2 SP2 is freezing daily as well. (6-8 hours) This is an install done by HP GDS.

The question is which version is stable?

5.1 SP1
5.2 SP1

I've been seeing in the forums that 5.2 SP2 and 5.3 are both unstable.

Survey says...?

[Darrin Rawls]

Is this just base HP SIM or is it full ICE? Is everyone seeing this in a VM?

[Rob Buxton]

I do have the occasional issue with our 5.2.2 version freezing, it's usually DB locking.
5.3 has a number of issues here. Freezing up being the main one. That seems to be a Java issue.
5.2.2 is more stable in my view.

[Russ Todd]

My configuration is not a VM. Physical Hardware.

I am still seeing the issue. My system was completly stable using 5.2.2

I am using SQL server 2005 with SP3.

This is a standalone HPInsight only setup, no ICE.

[Gus Tonsay]

It is on a physical server. The setup is the entire ICE package, brand new install.

HP BL460c
WIN2K3 32 STD
4GB RAM
SQL 2005
SIM 5.2 SP2
RDP 6.8 (378)

[Darrin Rawls]

Okay, if you can make sure there are some support cases that will help. On a problem server, if you can run a simcapture via the command prompt and email it to me that would be good (provided its not over 10MB).

[Russ Todd]

How big would you expect it to get in 1 day? I've got 400 + systems being monitored.

[Darrin Rawls]

simcapture will pull together several logs and place them in a zip. Depends on older logs that would still be around (is the major delta point); else it would be under the limit.

[Gus Tonsay]

Ran simcapture in ..\Program Files\Systems Insight Manager\lbin\simcapture

Emailing simcapture.zip

[Darrin Rawls]

uh, send it to me :)

[Gus Tonsay]

Correction

[Russ Todd]

Mine also has been sent

[Scott White_5]

Same problem here - dedicated system running SIM + some ICE components (VMM, PMP, IPM).

DL380G5 4GB RAM W2k3 Std 32bit
Remote SQL 2005 SP3 DB.
Monitoring 2400+ systems

I seem to make it about 2 days before things go south, typically JAVA HEAP errors. Restarting SIM fixes the problem. Sunday I logged on and it was hung, mxdomainmgr.log was 2.3GB!? Also see headdump.hprof files of about 1GB in the logs directory every couple days - is this an actual SIM dump file?

Forwarding my simcapture also...

Thanks,
Scott

[Darrin Rawls]

I think we have enough captures to triage; thanks!

-Darrin

[Anky]

did you get any solution for this error. has HP soem workaround for this?