Missing Log Message in SNMP Trap details

[Brandon Pembo]

Problem description :
I am running IpSwitch Whats Up gold V11 Network Monitoring software. And I am attempting to monitor my HP Proliant servers. I have the agents installed on the server as they were installed in the Pro Pack 8 I installed. When I pulled a Hard Drive out of the server, it sent the trap 3034 to the
monitoring system, however, I do not believe I am receiving the full trap. This is what I am receving as payload with the sent
trap:

TrapName=compaq-3034
TrapMajor=6 TrapMinor=3034
1.3.6.1.4.1.232.3.2.3.1.1.4.2.1=5
cpqDaLogDrvCntlrIndex.2.1=2
cpqDaLogDrvIndex.2.1=1
CommunityName=tpbp-pub
cpqDaCntlrHwLocation.2=Slot 0
cpqHoTrapFlags.0=12
1.3.6.1.4.1.232.11.2.11.1.0=12
Packet Type=Trap
sysName.0=BRCITRIX-A
Protocol Version=SNMPv1
1.3.6.1.2.1.1.5.0=BRCITRIX-A
Timetick=1days 17:53:31.40 cpqDaLogDrvStatus.2.1=5
1.3.6.1.4.1.232.3.2.2.1.1.20.2=Slot 0
1.3.6.1.4.1.232.3.2.3.1.1.1.2.1=2
Object=1.3.6.1.4.1.232 (compaq)
1.3.6.1.4.1.232.3.2.3.1.1.2.2.1=1

Ipswitch support informs me that I should be getting some more descriptive text from my Proliant server in that trap's payload. According to the MIB, as well as the HP SNMP Traps reference guide, I should receive a message to the effect of:

Logical Drive number %5 on the array controller in %4 has a new status of %2.

As you can see from the payload I pasted above, that text is not being sent from the Proliant device. I called HP support Friday, and I was told to call Microsoft. Microsoft has informed me the data being sent via Microsoft SNMP service is reliant on the sub-services (Agents) provided my HP.

Does anyone have any idea why the HP SNMP agents would leave out the log message. This message is vital to our monitoring. Knowing there is an issue with a disk drive in my array is great, but unless I know which drive...and what its exact status is, it's kind of useless.

I placed the HP Provided Mibs on the Ipswitch server. The Mib in question is cpqIDA.mib.

Where does that log message get generated? On the Proliant box and then delivered with the trap? Or should the monitoring software (IpSwitch) read that trap, reference the MIB and translate it to plain English?

I have called the tech support for IPswitch (the monitoring software in question) and they insist that the message should be sent in/with the trap from the Proliant server's SNMP agents. But this seems like it would negate the purpose of having a network monitoring software.

Please help,

Thank you,

Brandon C. Pembo

[Roy Main · Accepted solution]

A couple of ideas... perhaps you don't have the right MIB compiled into IPswitch? There are several drive related HP MIBS. Compile them all into your mgmt software.

You could double check which one is the correct MIB by looking for the 3034 trap in the MIBs you have.

Also, have u tried other mgmt sw to see how the trap is received?

Install Wireshark and capture the trap - it will show you the entire contents of the trap right off the wire w/o having to wonder if your mgmt sw is working.

[Brandon Pembo]

First and foremost, thank you for taking time out of your day to reply. I appreciate the effort.

I do believe I have the proper trap set in the Ipswitch management software. I was able to import the specifiec trap (3034) in to the system. I can also find all of the referenced avtive monitors from that trap in the management system. According to IPSwitch I don't have to "compile" the MIB, "supposedly" I simply have to place the MIB in the MIBS directory. It works fine for active monitors, but won't translate the traps properly.

I used your idea with the wireshark software. It showed teh traps being sent in their raw form. Basically the Oid's and their values. The trap and the Oid's are making their way to the Ipswitch software. And the IpSwitch software appears to be translating the Oid's into the proper strings. for example it might translate 1.3.6.1.4.1.232.3.2.3.1.1.4.2.1=2 in to cpqDaLogDrvStatus2.1=2. That's not my issue. My issue is that according to the HP documentation, I should also be recieving a nice little message that says

"Logical drive 1 on port 0 has a status of failed."

I'm not receiving this message. I'm new to this network monitoring stuff so please excuse my ignorance, but shouldn't IpSwitch see the trap, and reference the MIB to derive that message? There is note of that message in the MIB. I see the following:

DESCRIPTION
"Logical Drive Status Change.

This trap signifies that the agent has detected a change in the status of a drive array logical drive. The variable cpqDaLogDrvStatus indicates the current logical drive status."

--#TYPE "Logical Drive Status Change (3034)"
--#SUMMARY "Status is now %d."
--#ARGUMENTS {5}
--#SEVERITY CRITICAL
--#TIMEINDEX 99
--#VARBINDSEVERITY 2

::= 3034

But I get none of that from the IpSwitch management software. I installed HP SIM temporarily, and I get a legible English explanation of the trap from them. I think HP wrote these MIBS specifically for their own management software. Possibly written in a form that would make it a nightmare to get them to work properly with 3rd party softwares.

Thank you

BCP

[Roy Main]

Right, what you see on the wire is what the agent is sending in the trap.

I looked at the MIB - you're right, it's in the cpqida MIB. You will get 6 variables in the trap (along with some default stuff) - none of them is the string you are looking for, "Logical drive x.....".

What that means is that it's now up to your mgmt software to build an intelligent message for you from the data it is receiving. I'm sure the HP trap reference guide is documenting from the perspective of using HP agents with HP mgmt sw.

It's not that HP is doing anything to prevent 3rd party sw from working - they have just enhanced traps coming from their own hw.

[Brandon Pembo]

Again, thank you VERY much. This is what I have suspected all along. but with IpSwitch, HP, and Microsoft all pointing the finger at each other I wasn't 100%.

So if I understand the process correctly, the trap is sent in a very vanilla Oid=value format. Then it is essentially the management software's responsibility to receive said trap, reference the MIB and/or CFG files and forward me an intellegent alert. so untimately, the ball is being dropped at the management software's level. Of course they are going to suggest that the HP Mib is no good.

In addition to the cpqIda.mib, there is also a cpqIda.cfg file that I have found. This appears to be what the HP mgmt software is referencing in conjunction with the Mib file to derive its intellegent message. apparently the IpSwitch software isn't advanced enough to incorporate the CFG file.

Thanks again...You've given me more direction in two posts than all the support tech's combined. Points coming your way...

Brandon Pembo

[Roy Main]

Yes, if you are the mgmt sw you can look at the MIB and see what is (theoretically) available in the agent.

Now you can go back to the agent (if you need to) and try to do something intelligent. SNMP provides for this. However, depending on the quality of the MIB and agent, this task may or may not be difficult... ;-)

As a mgmt application you don't have to use what's in the trap. You can improve it or leave as is at your discretion.

[Brandon Pembo]

The HP agents don't appear to be very configurable. I can set server role and trap destinations, but that is about it.

I guess the message will need to be generated somehow from IpSwitch. I'd assume via a VB script. I guess I'll spend the next week learning VB...lol.

Thanks again for your help. It is appreciated!