iLO Firmware upgrade brings down VMWare server

[Thomas Harder]

Has anyone seen the following...we upgraded iLO FW to version 1.82 on a DL580G2 running VMWare ESX 2.1.1 build 9157. Shortly after the upgrade we received the following 4 HPSIM messages and then found the VMWare server locked up and had to power cycled via virtual power.

Keyboard Cable Disconnected. The Remote Insight keyboard cable has been disconnected.

Mouse Cable Disconnected. The Remote Insight mouse cable has been disconnected.

Remote Insight/ Integrated Lights-Out Self Test Error. The Remote Insight/ Integrated Lights-Out firmware has detected a Remote Insight self test error.

Remote Insight/ Integrated Lights-Out Interface Error. The host OS has detected an error in the Remote Insight/ Integrated Lights-Out interface. The firmware is not responding.

If the iLOs are suppose to be completely separate from the OS we're puzzled as to how this could be happening however, we have 3 cases now where this has happened so I don't think it's coincidental.

We used the Lights Out Migration Utility to upgrade the iLOs since it gives us the ability to do multiple upgrades at one time.

[David Claypool]

Get this called in to customer services and documented with a case number so it can be escalated.

[Uwe Zessin]

The iLO runs separate from the OS, but it is in the data path for keyboard/video/mouse.

[Thomas Harder]

I have a case number for this issue now....waiting for their response.

HP support # 3213072780
VM SR Id #224349

I forgot to mention that we currently don't have a mouse or keyboard plugged into this DL580 however I don't think this should matter.....we've upgraded hundreds of DL380Gx models running Windows without mice or keyboards plugged in and had no issues.

[Thomas Harder]

I opened a case with VMWare to see if they've experience this problem and here is there response.

"We have seen cases in the past, that upgrading the firmware on the iLO card
of an HP server can cause ESX server to crash. The iLO card sits on the PCI
bus, and re-flashing the firmware for iLO may require the iLO card to re-scan
and or re-assign the IRQs for the iLO card. We recommend scheduling ESX
server down-time in order to do any flashing of firmware / BIOS on hardware.

I hope this has answered your questions.

Take Care,
"

[Rich Colbert]

I'd throw in a vote that both the iLo behavior is likely to blame, but also a nod to considering upgrading to ESX 2.5.2, which in our experience has been the best and most stable version to date.