Automatic Event Handling no e-mails

[Bob Zabaga]

Okay, thanks! I'm 6 hours behind you! ....East Coast U.S....have a good night.

[Andrew_Haak]

The info seems conflicting but as mentioned before you can have a server that has an error on the hardware status and no event. If the event was cleared and the error was not solved the polling shows that the error remains. This is one option, the other is that even if you don't have traps configured. If your server uses WBEM you have to tell the server where to send it's events to. So let's start with one server. Select your server in SIM. Go to Options > Events > Subscribe to WBEM event. Now select Next and then Run Now. Let this job run with an account that has admin rights on the destination. Now that this job should have finished you have told the server where to send his events Now log into the server that you just configured with the System Management Homepage, Go to Settings > Test Indications > Send Test Indications. Select a type of events that you want to test (1,2 or 3) and push the next button. This should send the type on event you just selected. Now look in SIM and click on the ES link for the server you just created the event(s) for. This should not be green by the way if the test event(s) already arrived. There you should see a Test Event that is not cleared. If the events is cleared there should be an Automatic Event Handling job that has cleared the event. After the event is cleared the ES (Event Status) the status color point should go back to green. SIM has buildin jobs that will delete events older that 90 days. So if the event is deleted and the problem is not fixed the event is gone but the hardware status stays degraded. One small note for the near future. Gen 8 and above should be monitored by the ILO and Agentless Management Software. But just for now to understand how SIM works lest stick with the WBEM monitoring. This is the best way for Gen 7 and below on a Windows or ESXi 5.5 platform.

[Bob Zabaga]

Hello.  Okay, I will investigate and get back to you. Thanks for your suggestions.

In the meantme, I can tel you I made some amount of progress overnight.  I reconfigured SNMP on one old test server (where I can pull a drive). Then I restarted SNMP and the dependent HP services on that server. NOW - that machine shows both the memory and drive errors on SIMs main status page AND on the EVENTS tab for that server. That is new, didn't work before. BUT - no e-mail was sent.  I'll attach 2 screenshots.  I believe I'm monitoring this server with both SNMP and WBEM.

So let me get to your e-mail above and try that stuff and I'll let you know what the results are. 

take care,

Bob

[Bob Zabaga]

Hi again.  Aside from my prior post, I went ahead and ran your test.

From the HP System Mgmt Homepage on my test server "eddydev03" I sent 2 warnings.

Then I went to HP SIM and saw the events had appeared. 

But my original problem remains that I don't get an e-mail - for the test events or the bad memory or pulled drive.

I do get some emails when servers report errors but mostly no. These e-mails come from HP SIM, correct, not from the individual servers I'm monitoring...

You can see by my screenshot that my test alert sent from my test server (eddydev03) was picked up by HP SIM.

Just no e-mail.

-Bob

[Bob Zabaga]

Hey, one more thing! I see what you mean when you said Gen8 and Gen9 servers should use HP Agentless management. I read that section in the SIM guide. In iLO, there's an an entire SNMP section. My question is : do I use this in place of the standard SNMP service on the server or in conjuction with it?? Having SNMP in my iLO and configured in windows services seems redundant. So, between iLO SNMP, regular windows service SNMP and HP WBEM should I use all or some combination thereof? Wow, I feel like I should apologize for the stream of e-mails. I owe you at least a tremendous THANKS!

[Andrew_Haak]

Hello again,

Yes for the Gen8 and higher you should use the AMS in combination with the AMS ILO settings. If needed i can tell you more about this and how you can set this up. For other servers you can use the methode we use at this moment , agents using SNMP or WBEM traps. One thing to keep in mind use either AMS or wbem/wbem agent nog both. This will give problems like datacollection issues. Also you can have problems when you use both wbem and snmp agents on the same server. WBEM is prefered as this is what HP keep supporting in favour of the older SNMP agents on Windows servers. Gen8 and above has agents for windows, esx and linux, the real advantage of AMS monitoring is that you don't need OS credentials and thus i more save as you only need SNMP read rights.

To get back to the e-mails you don't get, look at the automatic event handeling and look for a email event handler that was triggered when the alarm was received. Can you see if that job uses email forwarding and if all settings seem ok. I always add the clear event trigger so i can see if the event was handeld or not.

Did you create your own email forward rule? Please writd the settings for this rule in the post. If the email server is comfigured you should also be sble to send a test email from the email settings.

We are almost there ! And don't worry about the number of messages i just like to help as you might have seen in this forum.  

[Bob Zabaga]

Hi Andrew.

I get emails sometimes. When I go to Options -> events -> automatic event handling -> "e-mail settings" and pick "send test e-mail" it works every time. I don't see why SIM shows critical and major alerts that I don't get e-mails for...Automatic Event Handling is configured to collect on "Important Events" (which is critical, major)....and set to collect on "All Servers" & send an e-mail with no time constraints. All looks good to me. I even deleted the event handling settings and re-created myself to be sure. But the prior settings looked fine.

I thought AMS was just in iLO. So with GEN8 & 9 I have 3 choices? AMS, SNMP, WBEM? I guess I can individually configure servers to use one polling method or another - not sure where to set these individually but I can go thru the documentation. On one of the general config page settings, there's a spot where you configure ALL agent monitors - that's set. But per server, I'm not sure where to do that just yet. I think I do have some that are set for both SNMP and WBEM.

If you view my sceenshot you'll see what I find extremely confusing. This is just one of many examples. I think I showed this in a prior post. Main HP SIM page show a major alert on a machine called sgesx01. If I click on the machine in SIM, system tab, I see the "health status" network alert (orange "!"). On the events tab it doesn't show as an alert. If I go to iLO on that server and examine th NICs, there's no problem. So why is SIM showing that alert on the main SIM page and on the system tab for that machine? I can't use SIM to manage my hardware if I'm getting false alerts or mixed messages like that. Could this could be why I believe I'm not getting e-mails for alerts? My assumption is that if SIM alerts me on the main system status page and shows that NIC alert again on that server's system tab, I should get an e-mail. But on the other hand, the same alert isn't shown on the events tab or in iLO on the server itself. So is it a problem? Should I be getting an e-mail? Who knows?!

I have a lot of servers I need to monitor, automated as much as possible and it's just not working for me. I'm really discouraged with this product! 

[Bob Zabaga]

Hey again....just adding an example to my last e-mail. See screenshot.

On FRI 18th, I added a new server "sgesx15" to be monitored. That seemed to work. I think its working....?

I'm confused because when I view "uncleared events" I see an Alert for sgesx15 that says "data collection task could not find any data to collect....taget system may not be running snmp, wbem, etc". 

That target system does have SNMP and WBEM running on it.  And if I click on the server in HP SIM you can see it reports back "Health status" as being monitored by WBEM and shows all hardware in green.

you'll see this in the screenshot. For me, that's sending another mixed message....Why would I get that warning on SAT 19th about "no data to collect" yet it seems to be using WBEM just fine?

-Bob

[Andrew_Haak]

let's try to make things clear: On in a Gen 7 and lower you should alway use WBEM on Windows server, SNMP on Linux and WBEM for for ESX 5.5 and above. On a Gen8 or higher you should use the SNMP/AMS setup. Disable any other agent, SNMP, SNMP agents on the operating system. AMS only works right when all other agents and OS SNMP are disabled. If you set the ILO, be sure to set the following SNMP Read & Trap community and trap destination to the SIM server Set the send traps using the hostname instead of the ILO name. If you get datacollection erros it's probably with the Gen8 or higher using any other than the AMS/ILO setup. Datacollection is a job that collects information for the servers in SIM. It has nothing to do with monitoring functions. The two things SIm does is receive traps SNP or WBEM and datacollect status and hardware inventory. where do you want to focus on right now ? If monitoring is the main thing (i would imagine) ignore the datacollection timeouts for now, we can fix that as soon as the email function is working. If you look at the left and you look at the Event collections you should find the Collection used for emailing. So all events found in this collection should create an event. I made a selfmade collection for Minor, Major, Critical events. The screenshot you show shows a NIC problem on a server. Go to the servers SMH and see what is wrong. The most confusing thing HP has done is create SNMP, WBEM and ILO events and they don't always match. So stick with the protocols mentioned above a the top and only use one protocol at a time for each server type.

[Bob Zabaga]

Okay, thanks I'll get back to you. That machne with the NIC warning is an ESXI 5.0 host. I'm either monitoring ESXI 5.0 hosts or Windows 2003/2008R2 machines.....G9 down to G5 maybe. I'd love to see those ESXI hosts upgraded to at least 5.5 but that's out of my control.