Writing software to pick up state of Integrity server front panel health LEDs

[Max Bruno Jarvis]

We have some rx2660 Integrity servers. Does anyone know how to write software to pick up the state of the front panel system health LEDs, or the LEDs on the Insight Display? I think it should be possible because I’ve read about utilities that seem to do it, but I need to do it in self-written software so as to raise an alarm in a way that’s specific to our application.

[Hein van den Heuvel]

Have you seen the iLO HTTP interface to the console? Use that?

Use SNMP?

I suppose you could script a telnet or ssh and ask for the VFP Command output.

None of this has anythign to do with OpenVMS of course.

You are adviced to tackle this problem from a generic, non-OS centric, hardware angle.

Just Google for RX2660 and SNMP and take it from there with topics like:

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=115&prodSeriesId=3346452&prodTypeId=15351&objectID=c01918605

Good luck!
Hein

[Max Bruno Jarvis]

Sorry, I should have mentioned that our rx2660s run OpenVMS, v8.3-1H1.

[Hein van den Heuvel]

>>> Sorry, I should have mentioned that our rx2660s run OpenVMS, v8.3-1H1.


Why does that matter what OS it runs to check out the HW health?

Unless you want it to monitor itself ?!
In that case, how will you detect fatal problems?

fwiw,
Hein

[Max Bruno Jarvis]

I was hoping someone would say "Call OpenVMS system service so-and-so" or "Use this OpenVMS utility" or something of that sort.

[Craig A Berry]

There is a Net::ILO Perl module for talking to the lights-out manager:

http://search.cpan.org/~nlewis/Net-ILO-0.53/README.pod

I had a quick look at building it on VMS once, but it had some dependencies that I didn't manage to get working quickly. However, while monitoring from another VMS system makes perfect sense, the path of least resistance might be to monitor it from something else.

There is a manual about iLO scripting at:

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00294268/c00294268.pdf

but it appears the only supported clients run on Windows.

[Shriniketan Bhagwat]

Hi,

Below links may be useful.

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=ru&cc=ru&taskId=120&prodSeriesId=3346452&prodTypeId=15351&objectID=c01230799

Refer chapter 5 from the below user service guide.

http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01404139/c01404139.pdf

Regards,
Ketan

[Martin Vorlaender]

Max,

as the document that Craig pointed out states, iLO communication uses the IPMI protocol, which is an Intel industry standard. There are freeware tools for IPMI, e.g. http://ipmiutil.sourceforge.net/
I have no idea, though, whether these can be tweaked to build under VMS.

HTH,
Martin

[Steven Schweda]

> I have no idea, though, whether these can
> be tweaked to build under VMS.

Might need more than a tweak.

http://ipmiutil.sourceforge.net/docs/UserGuide

[...]
------------------------
2.0 DEPENDENCIES
------------------------

The IPMI Utilities will run on Linux, Windows Solaris, or FreeBSD, and should
be portable to other OSs, if an IPMI driver for that OS can be obtained.
[...]

I don't know what "an IPMI driver" does, but
if I have one, then it'd be news to me.

[Kris Clippeleyr]

Hi,
The only reference to IPMI I can find sofar, are the files:
SYS$COMMON:[SYSHLP]VMS831H1I_IPMIAPI-V0100.RELEASE_NOTES
SYS$COMMON:[SYSLIB]HPIPMI_API.EXE
SYS$COMMON:[SYSLIB]HPIPMI_API.H
SYS$COMMON:[SYSLIB]HPIPMI_TYPES.H

The HPIPMI_API.H file declares 4 "get"-routines, an "init" and an "uninit" procedure.

Not much, but maybe a place to start?

Greetz,
Kris (aka Qkcl)

[Martin Vorlaender]

> I don't know what "an IPMI driver" does,
> but if I have one, then it'd be news to me.

The ipmiutil homepage says

"If you are using the IPMI LAN interface with ipmiutil utilities (-N option), neither the local or remote system requires any IPMI driver."

So, if you configure IPMI LAN access in the MP LAN settings, and can connect to the MP LAN interface, it should work without an IPMI driver.

But the Shareable and the header file Kris found look interesting. I'm just now having a look...

cu,
Martin

[Max Bruno Jarvis]

Thanks for all suggestions. A bit more explanation: these servers live in a closed cubicle where nobody can see them, running a monitoring system for power distribution. What I was looking for was an easy way to say to the power system controllers "Hey, one of the little LEDs on the front of the server has come on! Maybe one of its fans has failed, please have the technicians take a look". Failing that, I will just tell the technicians to open the cubicle every so often and take a look. The servers have their consoles on the serial console port and don't use the iLO2 LAN port, so I guess that would be a possibility but from what I see it would be more work than it's worth. Thanks again for your help.

[Jeremy Begg]

Hi Max,

An interesting problem, but maybe you're asking the wrong question. What you really should be asking is, "Do rx2660 hardware failures, such as a fan failure, get logged into ERRLOG.SYS? And if so, is there an OpenVMS-supported mechanism for reporting them automatically?"

(The WEBES/SEA stuff used to do this on Alpha, I don't know if it will do this on Integrity.)

Regards,
Jeremy Begg

[Brian Reiter]

If you can put the IlO MP netowrk ports on the LAN, log into it and issue the VFP (Virtual Front Panel) command.

I set up our RX2660 so that the MP ports were on the cluster LAN (separated off), I can log in via SSH or TELNET go do maintenance. This preusmably could be scripted and the output parsed.

[Ian Miller.]

I would expect that the WBEM stuff or ELMC would report the sort of hardware issue that would cause one of those lights to be lit.

[Hoff]

You're not going to get a pretty lights display nor a traditional OpenVMS system service interface (!?) nor anything even resembling the general approach you've requested here, and you probably don't really want that anyway, as (when these Itanium boxes do fail, and all marketing around "RAS features" aside) hardware failures will tend to wedge or even crash the host OS. (There are some cases that won't crash and burn, but those cases are usually not where you want sound the klaxons.)

It is usually best to receive and process iLO or iLO2 SNMP traps from a remote server.

Configure the iLO device to send SNMP traps to a server on your local network that you're using to coordinate these activities. This server will be running Nagios or some other SNMP software tool.

You can have up to three target servers, IIRC.

Nagios plug-ins associated with HP devices are here:

http://exchange.nagios.org/directory/Plugins/Hardware/Server-Hardware/HP-(Compaq)

The target server here (and again, running Nagios or otherwise) usually isn't VMS. Use of Unix here is probably easiest; this stuff all just works, and you'll have more of a tussle getting VMS to work.

If/when you wade into the world of iLO- or iLO2-generated SNMP traps, make sure the iLO and iLO2 firmware is current, as earlier versions of these devices can skip sending out some traps for certain significant events.

For the reverse, there's also some Perl code around that uses XML to allow a remote host to query the iLO2; see:

http://exchange.nagios.org/directory/Plugins/Hardware/Server-Hardware/HP-%28Compaq%29/check_ilo2_health/details

IIRC, some of (all of?) the in-bound SNMP stuff is pass-through, and the in-bound SNMP message traffic ends up getting passed upstairs and into whatever Insight Management agents are running (and they must be running) in the host OS. Put another way, iLO and iLO2 don't seem to have their own SNMP implementations, which means SNMP "discussions" with a semi-dead server are a non-starter.

The iLO and iLO2 ITRC forum is here:

http://h30499.www3.hp.com/t5/ITRC-remote-lights-out-mgmt-iLO/bd-p/itrc-298

[Hein van den Heuvel]

>>> What I was looking for was an easy way to say to the power system controllers "Hey, one of the little LEDs on the front of the server has come on!


And that's the answer I gave you to begin with.
Have them point a web brower to the ILO port and voila!

Hein

[Robert Gezelter]

Max,

Reading the entire thread to date, I would agree with the premise: Bring back the MP LAN connection. Of course, on a separate LAN. One then has the full virtual front panel display (in effect, the "light" can come on anywhere that the console MP can be reached).

- Bob Gezelter, http://www.rlgsc.com

[Max Bruno Jarvis]

Here's what I've decided to do: connect the MP network port to the LAN, and write OpenVMS software to log in to this port every so often and use the PS command, which returns data about temperature, fans and power supplies. Of course, this won't detect any failure which stops the rx2660 in its tracks, and we have another way of doing that. On this site the users wouldn't be allowed to log in with a web browser, and that idea also has the disadvantage that it relies on them to do something, whereas I need a way to detect trouble automatically and sound an alarm, without any action by anyone. Thanks to all for your suggestions. Max.

[Max Bruno Jarvis]

Have now written this software, seems to work OK.