topic Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines) in Server Management - Systems Insight Manager

Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 14:43:28 GMT

Server Specs:
HP ProLiant DL380 G4
Processor1 Intel XEON at 2.800 GHz
Processor2 Intel XEON at 2.800 GHz
4GB of RAM
Windows 2003 Std
MS SQL2005

SIM Software:
Operating system: Windows 2003
Version: Systems Insight Manager 5.1 with SP1 - Windows
Build version: C.05.01.00.02
Build date: 2007-04-11 17:26
Patches: HOTFIX51_001.jar Wed, 1/31/2007, 10:28 AM EST
HOTFIX51_002.jar Wed, 1/31/2007, 5:24 PM EST
HOTFIX51_003.jar Fri, 2/9/2007, 12:08 PM EST
HOTFIX51_005.jar Tue, 4/24/2007, 6:23 PM EDT
HOTFIX51_008.jar Mon, 7/2/2007, 3:57 PM EDT
HOTFIX51_011.jar Wed, 8/29/2007, 5:49 PM EDT
HOTFIX51_012.jar Fri, 9/7/2007, 5:28 PM EDT
HOTFIX51_015.jar Mon, 10/8/2007, 2:12 PM EDT
HOTFIX51_016.jar Mon, 10/8/2007, 2:18 PM EDT

Plug-in applications:
HP BladeSystem Integrated Manager 2.3
HP Insight Power Manager 1.0
HP Performance Management Pack 4.3
HP Server Migration Pack 2.0.2
HP Virtual Machine Management Pack 2.0.2
Remote Support Configuration and Services 1.0

SIM Environment:
2742 Systems discovered (1078 identified SERVERS)

Issue:
SIM performance is incredibly slow on and off and I cannot put my finger on the main issue. At first SIM was almost inoperable then I found that if I change the schedules for the tasks below to once a day it did not lock up SIM as bad:
Hardware Status Polling for non Servers ran every 10 mins (And on start up of the CMS)
Hardware Status Polling for Servers ran every 5 mins (And on start up of the CMS)

This is now set up to be removed from the list of possible culprits.

With that done SIM was then somewhat "managable" but if a task was kicked off it would lock up SIM and I could not navigate or refresh the web page at a rate that I can say is effecient. It would take over 10+ mins to refresh or navigate to a page.

This is still the case and I am almost at a point to abandon the entire project.

mxdomainmgr.exe is always suspicious as its using 1,600,000K of memory and 100% of one of our CPU cores. This usually is corellated with a task being launched. Read some forum articles but nothing to help me.

I AM LOOKING FOR HELP on speeding up performance and\or figuring out the root cause so I can determine whether or not we should move on. We are planning on pruchasing the essentials packs for all our servers IF this is resolved!

I'm working with HP in trying to get a backend engineer to assist........

Signed frustrated and disapointed!

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 14:57:28 GMT

Cavalarie to help..tu tu tu tu tu ;)

Ok number one, you need to leen down some item,more information is adding up in you SQL data thta is is not needed.So you do not need all informational about you'ree systemes.
HP Insight menu >Events>Event filtering setting> Check Accept registered events with severity:
Check these Critical,Major,minor and Warning, DO NOT CHECK INFORMATIONAL.

PS:Don't ferget to apply you're points,there are situated next to every messages in these thread,next to every messages date and time has UNASSIGNED.

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 15:05:53 GMT

Step #2 is to remove all informational date in you're server HP Insight menu Option>Events>Delete Evants

Selete >Informational Events..APPLY>RUN NOW

This reduce you re database alot and then you should a difference.

Dan

PS:DOn't ferget to apply you're points in every messages in these thread,has UNASSIGNED

WHY SHOULD YOU DO THIS?
There are 3 reasons why we feel rating replies is such an important feature:

1.) Others have taken time to help you, so please give them credit for their help.
2.) Your rating will help your peers earn points toward their Forums status, and you will validate the quality of the solution you've received.
3.) Other readers will understand which answer best solved a problem, enriching the community knowledge being shared

Thanks you

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 15:20:33 GMT

Thanks for you tips, not sure if it will help my main issue but it is DEFINETLY a good modification as I am sure there are thousands+ of informational events due to the size of our enviroment. It is surely an excellent suggestion. For this I would give a full 10 points under tweak\tip for overall SIM configuration mod.

Not confident it will be the reason for our performance issue but if it turns out to be I will make sure to update this post.

Any other particular changes to minimze db transactions?

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 16:21:23 GMT

Not sure if because the little rabbit coming out of the hat icon will mean this is resolved or not but to be clear its not... :) thanks

"Magical answer Message with a response that solved the author's question"

Not the case as got some good mods but not the actual fix for my problems.

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 17:49:56 GMT

Do you try it to see what it did anyway?

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 18:05:15 GMT

Made the changes you recommended but still rather slow....
Going through the list to see if I can filter some discovered systems currently I have the following break down:

Enclosure=98
Cluster=2
Management Processor=729
Printer=2
Rack=33
Server=1017
Switch=2
Tape Library=5
Unknown=362
Unmanaged=518

If I filter out Management Processor from being discovered will the servers not see its ILO associated with?

I can't remeber where it was to choose what to keep or not keep depending on the type of discovered system.

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 18:13:41 GMT

Did you delete all informational ?

There also information older then a certain time you can delete.

if you filter ILO it will be able to associate them.

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 18:19:36 GMT

I believe it did, showed completed in the task results. How can I confirm manually?

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 18:34:38 GMT

Take a few server out of the blue and verify there events.

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 19:45:52 GMT

I have a lot of "Discovered System" events that I cannot delete..... any way to delete them. I get:

Could not delete all selected events.
One or more events are discovered systems
one or more events are in pending state.

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 19:54:13 GMT

ohnn oh...that not a good signe,usally everything should be cleared, you could delete(Server) and rediscover those items,try a few to see what happens.

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Wed, 12 Dec 2007 20:21:54 GMT

Just as a side note, noticed that there are 35000+ events older then 90 days.... and 100,000+ events in total. Will work on clearing these up first.

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Wed, 12 Dec 2007 20:27:40 GMT

Wow no wonder that why.

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Thu, 13 Dec 2007 15:31:12 GMT

So has it going?

Ps don't ferget to apply the other messages point in this thread Thanks you

HAve a nace day and happy chirstmas

Dan

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Thu, 13 Dec 2007 15:43:39 GMT

You know i just had a flash genius do you know the prgram HP SIM Sizer,i verfied if you have around a 2000 nods it select a ML370 G5

This solution includes:

Dual Core Intel Xeon processor
Intel Xeon 5060 (3.20 GHz, 1066 FSB)
Max processors: 2 x 3.16 GHz/ 1MB Cache
HP 3 GB Fully Buffered DIMM PC2-5300 Max Memory: 64-GB

Fault Tolerant Storage
Mirrored 36 GB SAS 15,000 rpm Hard Drive

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Daniel Leblanc — Thu, 13 Dec 2007 15:44:35 GMT

if you would like to verify :
http://h71019.www7.hp.com/activeanswers/Secure/274366-0-0-0-121.html

Re: Systems Insight Manager 5.1 with SP1 (with OSEM\ISEE) performance issues (1000+ managed machines)

Mariano De Luca — Tue, 18 Mar 2008 18:35:51 GMT

My issue is RESOLVED................. we went through multiple techs and suggestions on things to try but none were as useful as the following which was resolved by Steve (ISS Elevation Engineer):

Apparently my mxjobs table list was in the 100,000, so anytime I would attempt to navigate to tasks or results in sim it would simply hang (it would try to load but would never finish), clearing this table has returned all to normal!!!! Thanks HP!