Integrated Lights-out 3 interface status not responding

[SMaximus7]

I tested the beta drivers.  problem was gone for almost a month... however... it's back.  Did not fix the issue. 

[EverL0ng]

If their is any updates to this thread please keep us posted, we are having the same issues with our iLO3's (FW - 1.26 and iLO Drv - 3.3.0.0). This is driving our SCOM guys crazy with the 9006 alerts. 

Thanks

[MBSNL]

I logged the following issue with HP. As usual they don't seem to respond.

Because the ILO3 of a G7 Blade server was showing MAJOR in SMH/SIM, an engineer tried solving if by restarting ILO Management.

Result of restarting ILO Management caused Virtual Connect in the Bladesystem to -think- the server was powered off (it was not powered off). Result was that Virtual Connect remored all NIC connections to this Blade Server.

This Virtual Connect problem after ILO problems should (according to the knowledge base) already have been fixed in VC FW 2.3x

We are on VC FW 3.30... Help!

My problem is getting worse and worse... Is there some beta fix we can try?

[Pound30]

Any idea when 3.4 might be officially released?  We have thousands of servers and the ones with ilo3 for the most part have this issue.  3.4 doesnt seem to always fix it.  We need to get this fixed asap so we can monitor the servers correctly instead of reacting to false alerts.  This is a major issue to us even though there is no problem with the ilo itself.

[Oscar A. Perez]

iLO3/4 drivers v3.4.0.0 hit our FTP last week. They will hit the web later this month.

W2K8 x64 and R2.

 CHIF driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p1398020247/v67292

 Management driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p2122273788/v66427

W2K8 x86 and W2K3 x86.

 CHIF driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p1741131556/v67461

 Management driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p468919737/v66428 

[Steve Weeks_1]

I've just installed these on 7 machines.

They certainly clear the error and all work OK.

I'll update as to whether they resolve the issue long term.

[Oscar A. Perez]

After the fixes we added to driver version 3.4.0.0, I cannot reproduce this issue in my lab anymore. But, customers are still reporting it so, the case remains open.  

[zeroagemain]

Hi Oscar,
Is it possible to have an update on this issue?

I've been reluctant until now to add to the clearly long list of people complaining about this but at our site it is close to making SIM an unusable monitoring system because of the relentless frequency with which those doing the monitoring have to restart iLO3 to remove 'ilo3 not responding' alerts in SIM generated by HP Agents. We are on latest fware and driver combinations (1.28/3.6.0.0).

Regular info/updates on such issues are always appreciated, if nothing else we can show the poor souls doing our monitoring that it is being worked on.

Kind Regards,
Brett

[Oscar A. Perez]

We are about to release iLO3 version 1.50 within the next two weeks that comes with an overhauled CHIF subsystem that solved few known communication issues with the driver.  

I'm also investigating an issue with the ProLiantMonitor service that would randomly log an Event ID 78 then, Event ID 79 few seconds later.  This one has been very tricky to debug as it only happens once every 3 weeks in my lab setup.

[zeroagemain]

Thanks, we'll give that a try when it comes out.

Something I forgot to add to my original post; only Windows hosts are affected, Linux and ESXi hosts on G7/iLO3 don't give false positive iLO alerts in SIM but all of our Windows hosts on those hardware platforms do.

[Oscar A. Perez]

iLO3 version 1.50 was released to the FTP this morning. It should hit the web within the next two weeks. Please give it a try and let me know the results.

FTP links:

Online ROM Flash Component for Windows 64-bit

ftp://ftp.hp.com/pub/softlib2/software1/sc-windows-fw-ilo/p1728391553/v73058

Online ROM Flash Component for Windows 32-bit

ftp://ftp.hp.com/pub/softlib2/software1/sc-windows-fw-ilo/p901462262/v73059

Online ROM Flash Component for Linux

ftp://ftp.hp.com/pub/softlib2/software1/sc-linux-fw-ilo/p1255562964/v73060

Note: Please clear your browser cache after flashing firmware 1.50.  There were lots of changes made to the webGUI in this version.

[zeroagemain]

Oscar,
Sorry to say that's not made any difference. The problem still occurs and after a few days it even appears to be more frequent on the servers that were upgraded but that might just be coincidence.

Is there really no idea within HP about what's causing iLO3 to give false alerts to SIM after all this time? With this and the recent false VMWare NIC alerts (which are now thankfully fixed) our Operations team have all but given up on realtime SIM monitoring as it is littered with far more false positive alerts than real ones. It's just checked from time to time for disk and RAM faults which is a shame.

[Oscar A. Perez]

FYI, apparently VCA isn't showing the iLO3 CHIF driver version causing customers to only upgrade the iLO3 CORE driver, therefore missing the fixes to the CPQSMIF.DLL that have been introduced since version 3.4.0.0.  Please check what version of CPQSMIF.DLL you have on your systems.  

More details here.

http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/Remote-Insight-Integrated-Lights-Out-interface-error/m-p/6070003#M136344