No event on PS failure??!

[BuddyD]

Hi Folks,

SIM 7.3.0 running on a Windows 2008 VM in a 3 node cluster ( Proliant DL380 G7). When we remove a power cord form one of the nodes (testing), the health status of the node recognizes we have "lost" a PS but no trap/event is generated. We are seeing the same behaviour with a 3800 switch.

What am I missing?

Thanks in advance for any/all help.

BuddyD

[Andrew_Haak]

Hello,

Did you configure SNMP or WBEM traps on your servers ?

You need to tell your servers etc, where to send their error to.

[BuddyD]

Hi Andrew,

Thanks for the reply.

The HP servers are ESX hosts configured with WBEM, which appear to be working fine as far as status of the various components. But....no alerts.  The ILO's are configured to send snmp traps to SIM via snmp. When I look at each host thru Configure -> Manage Communications, it thinks evry thing is fine, Identification and Events have a green check mark.

[Andrew_Haak]

Try to do a subscribe to WBEM events from the SIM interface. You can this option under options > Events > Subscribe to wbem events while the ESX servers have been selected. The SNMP forward trough ILO should also work.

[BuddyD]

Thanks again, Andrew.

Still no luck with the testing removal of a PS. I've been able to get acknowledgement of the "change" in both SIM(WBEM) and the Vsphere Client (alarms), but no alerts via snmp. However, if I log in to the ILO and send a test alert via snmp, it all works like a charm.

Where do I tell SIM the a "lost PS" is not an informational event and I want to be alerted? is there a WBEM notification system/config file somewhere?

[Andrew_Haak]

Hello again,

I know that snmp mibs severity can be changed from SIM but have not found an equal option for wbem events. You can configure the ILO to forward server events, these events would be SNMP and then you can change the severity of the event here :
Go to Options => Events => SNMP Trap Settings
find the corresponding mib and content, to change the status.

Original post : http://h30499.www3.hp.com/t5/ITRC-HP-Systems-Insight-Manager/Change-Event-Severity-of-a-specific-event/td-p/4706525#.VbKUOEpUeK0

[BuddyD]

Thank you again, Andrew.

That didn't seem to work. I change the severity to Critical but it still shows up as an informational event.

I've been reading thru a few other spearate posts here, a suggestion from Tushar Bajpai was to run mxwbemsub from the SIM host. When I do that and query my 3 ESXi hosts, I'm getting 2 Filter Names for each host, HPSIM_TYPE_1_hostname_0 and then hostname_1.  Although each query is different:

select * from CIM_AlertIndication and select * from HP_DeviceIndication.

Is this normal or problematic?

Further down in that same thread, there was an indication that disabling the ESX firewall helped. I'm headed down that path now...stay tuned.

[Andrew_Haak]

I've had a lot of problems with WBEM events for ESX servers do i switched to SNMP trough the ILO. Don't you get the event from the ILO then?

[BuddyD]

Thanks Andrew, I would have to agree with that statement. I've been able to solve the problem, bypassing wbem and going to snmp. I'm not sure why it still won't work from the ILO, but enabling snmp via ESX seems to have done the trick. The link I used to enable snmp is below, hopefully it will help someone else.

https://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.monitoring.doc_50%2FGUID-EA64297A-35FD-4226-A1B2-367C57D38CBD.html

I'd also have to agree with other folks in this forum who have experienced difficulty getting HP hardware to communicate properly with HP software ( SIM, IMC, etc.) I'm now struggling to get a few HP switch stacks to communicate with SIM and/or IMC.

I have 2 HP E3800 (J9576A) 48 port switches (stacked), SIM reports that it can successfully communicate via snmp but if I pull a power cord neither SIM or IMC report anything. Same for 2 stacks of edge switches ( HP A5120 )...frustrating.

Thanks again for the help and suggestions, Andrew.

I'll post the resolution if/when I get the switches reporting properly.

[Andrew_Haak]

Hello again, Did you configure SNMP trap to the SIM server on the switch ??

[BuddyD]

Yes. The switch stack sees both the SIM server and IMC server as trap destinations yet, neither reports anything when there is a PS event. The hetwork gear site behind our firewall so I don't think that's the issue.

As I stated earlier, I have run SIM discovery and it sees the stack and is communicating ( or so it would appear) via snmp.

And after typing that last line, I see that SIM received a link up signal from the swith stack on Friday when I was configuring things, however, I also notice errors in the details of that message. Looks like a Mib mismatch...stay tuned.

[Andrew_Haak]

One last tip then, Under Options > Events > Event Filter Settings you can change the settings Accept unregistered events. This should only be switched on for troubleshoot purposes, turn it of afterwards. If you don't you count get a lot of trash events where you don't have any MIBs for....

[BuddyD]

Thanks again.

I did that and it doesn't seem to recognize the OID coming from the E3800 switch. I've updated the MIB's on the SIM server, which I haven't done in a while. We'll see if that helps any.

[BuddyD]

UPDATE:

Installed the latest SIM mibs and re-registered them. I've opened up SIM to unregistered events and I'm getting complaints from HP switches ( E3800 and A5120 ), in addition to a Fortigate cluster. The complaints:

A genericUnregistered trap signifies that a system has sent a trap to the management server for a MIB that has not yet been compiled. Try compiling any MIBs from the vendor for this system to obtain additional information. See Details for more information.

Mib Information
The associated MIB File Name for this trap is cpqgen.mib and the MIB identifier CPQGEN-MIB

That's a Compaq MIB and with the latest HP SIM MIB pack installed, I'm confused. There doesn't seem to be specific MIBs for the E3800 or A5120, that I can find anyway.

I think I stated in an earlier post that the E3800 stack is being monitored by both SIM and IMC. If I remove power from one of the PS, I get nothing in either monitoring instance. It does show up, however, on the 3800 GUI interface (red).

I have a suspicion that the 3800's may need to be rebooted?!

Any/all hints and suggestions appreciated at this point.

Thanks.

Buddy

[BuddyD]

Added the Fortinet Fortigate mibs to the SIM server, recompiled ALL mibs and re-registered them all. Restarted the SIM service on the VM and here's a curious thing that I've noticed. Though the 2 Fortinet cfg files are in the mibs folder, when I look at SIM ->Options -> Events -> SNMP Trap Settings, I do not see the Fortinet MIBs listed.

Is this normal and/or expected behaviour? Did I miss a step along the way?

[Andrew_Haak]

Did you use the mxmib tool? Look at page 4 of this pdf guide http://www.hp.com/wwsolutions/misc/hpsim-helpfiles/simsnmp.pdf