ILO 2-4 Hardware trap to OpenNMS

[Marco Correnti Techno S]

I have many servers with ILO version "0" through 4. I need to send ILO hardware integrity traps to an OpenNMS system.

I did some tests by disconnecting the power cable, but only the Servers with ILO4 transmit the trap to the OpenNMS system (and not always ....)
It would appear that (Ref#1)  ILO 2 (and given the results I believe also ILO 3) can send only some traps, (Ref #2) without the "support" of the Operating System, is this my assumption correct?

In this case, what is the least "invasive" solution?

Thanks in advance

Ref #1  https://community.hpe.com/t5/ProLiant-Servers-ML-DL-SL/ilo2-to-send-snmp-traps-about-hardware-raid-power-etc-without/m-p/4578330

Ref#2 https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00043538en_us#N10070

[support_s]

System recommended content:

1. HPE Integrated Lights-Out (iLO)

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Marco Correnti Techno S]

Updated the first URL.

[ManBha]

Hello,

Please do refer the below documents.

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00045595en_us

https://support.hpe.com/hpesc/public/docDisplay?docId=c03351063&docLocale=en_US

iLO3: https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=emr_na-c02774507 [ page 83]

iLO4: https://support.hpe.com/hpesc/public/docDisplay?docId=sd00001038en_us&page=ref-ams.html

Thanks.

[Marco Correnti Techno S]

Thanks ManBha

I have already found the documentation, what I need is a "simple procedure" in order to do what I need....

It seems, but I am not sure, that at least I need to install the rpm hpasm for ILO2 & ILO3 systems.

(I have installed it and the ILO fan/power/temperature sensors information are visibile from OS level)

https://support.hpe.com/hpesc/public/swd/detail?swItemId=MTX_2f1c0bc1c61b4e7b9fd48305bc

I don't understand if the HP SIM is needed or not and if the ILO2/3 related configuration must be done.

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00043529en_us

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00043538en_us

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00045585en_us

Anyway I cannot find a SIM version that is compatible with RH 4 (we cannot upgrade ..)

https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c05350888#N10283

EDIT Just found the versione for RH4 https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c04269694

Suggestions ?

[ManBha]

Hello,

Yes, this is the right version of SIM for RHEL 4.

 https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c04269694

Thanks.

[Marco Correnti Techno S]

Thanks ManBha

My question was : is it enough to install only hpasm in order to send the i. e. Power supply traps to OpenNMS?

The path will be OS - - > ILO2(3) - - > OpenNMS

Or i need additional software (SIM) and configuration?

Regard

[Marco Correnti Techno S]

In mean time ...

#rpm -ivf OpenIPMI-tools-1.4.14-1.4E.25.i386.rpm
/etc/init.d/ipmi start

#modprobe ipmi_devintf
#modprobe ipmi_si type=kcs ports=0xca2

#lsmod | fgrep -i ipmi
ipmi_si 37865 0
ipmi_devintf 13385 0
ipmi_msghandler 32297 2 ipmi_si,ipmi_devintf

#/etc/init.d/ipmi status
ipmi_msghandler module loaded.
ipmi_si module loaded.
ipmi_devintf module loaded.
/dev/ipmi0 exists.

#rpm -ivf hp-ilo-8.0.0-144.rhel4.i386.rpm
#/etc/init.d/hp-ilo start --> OK

#rpm -ivf hpasm-8.0.0-173.rhel4.i386.rpm
#/etc/init.d/hpasm start --> ALL OK (except cmarackd that is stopped)

/var/log/messages now contain Power Supply events
Mar 16 13:01:53 RF4 hpasmlited[18296]: CRITICAL: System Power Supply: General Failure (Power Supply 2)
Mar 16 13:01:53 RF4 hpasmlited[18296]: WARNING: System Power Supplies Not Redundant
Mar 16 13:02:03 RF4 hpasmlited[18296]: NOTICE: System Power Supply: General Failure (Power Supply 2) has been repaired
Mar 16 13:02:03 RF4 hpasmlited[18296]: NOTICE: System Power Supplies Not Redundant has been repaired

but still no trap, I suppose I need to configure the SNMP Insight Agents in some way ....

Anyway the ILO 2 SNMP/Insight Manager Settings is set to True for

• iLO 2 SNMP Alerts
• Forward Insight Manager Agent SNMP Alerts
• SNMP Pass-thru:

[ManBha]

Hello,

We might have to involve the OpenNMS vendor/ OS vendor for the configuration requirements to receive the iLO traps in OpenNMS.

Thanks.

[Marco Correnti Techno S]

Many Thanks ManBha for your support

The situation is the following :

ILO2 Test trap is correctly sent from ILO web interface and received by OpenNMS ; the same for PowerOutage & ServerReset generated when the system reboot.

The "missing" traps are the ones for Power Supply outgage (and of course other traps, like fan & sensor temperature)

In /var/log/messages, after the installation of hpasm & hp-ilo rpm package, there are entries about the power supply outgage, like these :

Mar 16 16:29:43 RF4 hpasmlited[14501]: CRITICAL: System Power Supply: General Failure (Power Supply 1)
Mar 16 16:29:43 RF4 hpasmlited[14501]: WARNING: System Power Supplies Not Redundant
Mar 16 16:29:53 RF4 hpasmlited[14501]: NOTICE: System Power Supply: General Failure (Power Supply 1) has been repaired
Mar 16 16:29:54 RF4 hpasmlited[14501]: NOTICE: System Power Supplies Not Redundant has been repaired

Moreover the info about Power Supply (and others sensors) are correctly seen by Operating System :

# hplog -p
ID TYPE LOCATION STATUS REDUNDANT
1 Standard Pwr. Supply Bay Normal Yes
2 Standard Pwr. Supply Bay Normal Yes

I suppose that the cause of the "missing" Power Supply trap problem can be the snmpd configuration or the SNMP/IPMI RPM used.

After the installation of hpasm rpm there is an "activation" step

==============================================================================
NOTE: In order to activate the software contained in this package, you must
type 'hpasm activate' as 'root' user.

that, among other things, do a reconfiguration of /etc/snmp/snmpd.conf adding the  dlmod directive

(Dynamically Loading Modules) 

cat /etc/snmp/snmpd.conf
# Following entries were added by HP Insight Management Agents at
# Wed Mar 16 15:30:17 UTC 2022
dlmod cmaX /usr/lib/libcmaX.so
rwcommunity private localhost
rocommunity public localhost
# ---------------------- END --------------------

Infact now in the /var/log/messages there are entries related to this "module" libcmaX.so

Mar 16 15:30:27 RF4 snmpd[13314]: cmaX: subMIB 1 will be sent on port 25376 to hp Advanced Server Management_Peer
Mar 16 15:30:27 RF4 snmpd[13314]: cmaX: subMIB 2 will be sent on port 25376 to hp Advanced Server Management_Peer
Mar 16 15:30:27 RF4 snmpd[13314]: cmaX: subMIB 3 will be sent on port 25376 to hp Advanced Server Management_Peer
Mar 16 15:30:27 RF4 snmpd[13314]: cmaX: subMIB 5 will be sent on port 25376 to hp Advanced Server Management_Peer

I want underline the fact that same processes of hpasm are not running, maybe this normal for this ProLiant DL380 G6 running Red Hat Enterprise Linux ES release 4 (Nahant Update

# /etc/init.d/hpasm status
Status of hp-ilo:
hpilo driver is loaded...
Using standard Linux IPMI device driver and hpasm-lite

ipmi_msghandler module loaded.
ipmi_si module loaded.
ipmi_devintf module loaded.
/dev/ipmi0 exists.

hpasmlited is running... [ OK ]
Status of Foundation Agents (cmafdtn): cmathreshd cmahostd cmapeerd

cmathreshd is running... [ OK ]

cmahostd is running... [ OK ]

cmapeerd is running... [ OK ]
Status of Server Agents (cmasvr): cmastdeqd cmahealthd cmaperfd cpqriisd cmasm2d cmarackd

cmastdeqd is running... [ OK ]

cmahealthd is running... [ OK ]

cmaperfd is running... [ OK ]

cpqriisd is stopped... [ OK ]

cmasm2d is stopped... [ OK ]

cmarackd is stopped... [ OK ]
Status of Storage Agents (cmastor): cmaeventd cmaidad cmafcad cmaided cmascsid cmasasd

cmaeventd is running... [ OK ]

cmaidad is running... [ OK ]

cmafcad is running... [ OK ]

cmaided is running... [ OK ]

cmascsid is running... [ OK ]

cmasasd is running... [ OK ]
cmanicd (pid 14203) is running...

Moreover we are using the following SNMP and IPMI  RPM, so not the ones from HP

# rpm -qa | grep -i -e snmp -e ipmi | sort
net-snmp-5.1.2-18.el4
net-snmp-gmd-5.4.1.A-GB4xx
net-snmp-libs-5.1.2-18.el4
net-snmp-utils-5.1.2-18.el4
OpenIPMI-1.4.14-1.4E.25
OpenIPMI-libs-1.4.14-1.4E.25
OpenIPMI-tools-1.4.14-1.4E.25

Suggestions of what to check or configure ?

Thanks in advance