Remote Insight/ Integrated Lights-Out interface error

[jgonzalez]

I am seeing a big number of these errors recently being reported by SIM.

The host OS has detected an error in the Remote Insight/ Integrated Lights-Out interface. The firmware is not responding.(ILO still functional)

I was reflashing the firmware, but it did not fix the issue. I have back rev'ed the firmware with no luck. Only one version, but still no dice. We mostly have ILO3, but we still have some ILO2 and ILO in house. Mostly showing up on ILO3 interfaces. Most of these servers are running the lastest Support Pack and Firmware. But even then we are still receiving these errors. Not sure at this point. Any ideas?

Thank you for your help.

[Dennis Handly]

What server model do you have?  What OS version are you running?

[jgonzalez]

DL360 G7's with Windows 2008 R2 Enterprise.

[Rob Buxton]

I'd also verify the BIOS FW is up to date, get the latest FW DVD and update everything to keep it in sync.

You might want to then look and verify that you have the latest PSP.

[jgonzalez]

Most of these servers are running the lastest Support Pack and Firmware.

[Julio_Ortiz]

I am also experiencing the same issue. Same model Proliant DL380 G7. Windows 2008 R2 and Storage R2.

All have latest FW, Drivers, etc. Has anyone found a solution to this issue?

I

[Olson_1]

same thing happens on a bl460g7 with latest 1.26 ilo fw.

[JasonWoerner]

I'm seeing the same issue. Periodic. Two servers: ProLiant BL460c G7. 1.26 firmware.

[Snak3eye5]

I am seeing the same issue as well. THree of my bl 460c g7 blades.

[Julio_Ortiz]

FYI:

HP acknowledge an issue while troubleshooting something else in two of our file servers. This is still going on in more than 5 of our G7 servers.

[Renjiv]

Hi All,

The below info might help you all.

(SNMP) Remote Insight/ Integrated Lights-out interface Error (9006)

-       Remote Insight/ Integrated Lights-Out Interface Error. The host OS has detected an error in the Remote Insight/ Integrated Lights-Out interface. The firmware is not responding.

-       Consequently to the 9006 SNMP trap a 9005 might be triggered as well.

-       When this happens, System Management Home Page will display under Management Processor box Integrated Lights-Out 3 as degraded Despite the error being shown, ILO will remain functional and accessible

Solutions:

This issue has been seeing most commonly on ProLiant servers running Microsoft Windows 2008 Server Enterprise Edition X64 and in some ESX environments. Updating Management Controller Driver to version 3.3.1.0 might not resolve the issue in some cases.

If there is no production impact on this issue, then the error can be ignored.

Workaround:

Youu may also try the below workaround:

From SMHP(System Management Home Page) click on the Integrated Lights Out 3 link and reset ILO from there. This would work for an undetermined amount of time and the SNMP trap might appear later on.

SIM, SMH, are not the root cause of this instead as mentioned above a new Management Controller Driver will be released sometime in the future.

Regards,

Renji

[TippaFew]

Is this a tech note from HP - could you post the link to the article? Unfortunately my managament do not believe me and require verification that this was posted by HP...

Thanks!

[Renjiv]

We dont have any Tech note from HP. This solution is from my experience and ILO3 Management Controller Driver to version 3.3.1.0 worked in few scenarios.

You may get the ILO3 Management Controller Driver from below links:

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=18964&prodSeriesId=4154735&prodNameId=4154847&swEnvOID=4024&swLang=8&mode=2&taskId=135&swItem=MTX-e89f2d6b66c242cd989e9611b6

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=4091412&prodNameId=4091432&swEnvOID=4022&swLang=8&mode=2&taskId=135&swItem=MTX-9072a587fdd04c6b9e3eb856e2

Regards,

Renji

[Snak3eye5]

Ok i have updated my smart start and firmware to the latest and greatest. I have tried cycling snmp service and this resolved the error for me. I also changed snmp trap in services to automatic.

[Snak3eye5]

OK i take this back. It keeps happening and was only a temporary fix.

[riperez]

Any update on this. This starting popping up on some of servers lately.

[haven564]

Um ya, ignoring is not a solution nor is resetting a viable work around. This issue has been happening for many months across the board for us. Logging into each server each time an alerts pops for this is not efficient. We really need to see a response from HP on this.

[Josh_00]

This issue has come up on basically every DL360 G7 and DL380 G7 that my organization has purchased.

We have updated drivers to newest, firmware to newest, and then have performed an NVRAM reset. The issue still crops up.

There was a blip in a version changelog awhile back (can't remember if it was the newest firmware or newest driver) that mentioned erroneous 9006 errors being generated, so I got excited. But it makes no difference.

When someone receives a resolution from HP, please post. Our customers/management don't approve of the "ignore it" resolution...

[Josh_00]

I received this response from an HP tech tonight:

"Our engineering team is currently testing a beta version of [iLO Management Controller Driver] 3.4.0.0 that will be released once the testing phase has been completed."

[Oscar A. Perez]

iLO3 drivers v3.4.0.0 are on the FTP site already. I'm told they will hit the web later this month along with PSP 9.0.

W2K8 x64 and R2.

CHIF driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p1398020247/v67292

Management driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p2122273788/v66427

W2K8 x86 and W2K3 x86.

CHIF driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p1741131556/v67461

Management driver: ftp://ftp.hp.com/pub/softlib2/software1/sc-windows/p468919737/v66428

Let me know if it solves the issue for you.  We have few customers reporting that it only makes the issue happen less frequently and even though with version 3.4.0.0 we cannot reproduce this issuein our labs anymore, we are still working on this case.

[Goutham_Sabala]

Hi Oscar,

Can you confirm is PSP9.0 is going to be released, from what I understant with the release of SPP in November 10, PSP development was going to stop at 8.70. Please correct me if Im wrong.

I ask this because Im concerned with PSP ( Proliant support Pack ) development for servers 2003 as many servers are continuing to use 2003 OS for various reasons , if their are new drivers available for server under 2003 platform , is cusotmer supposed to download individual driver and then update each of them ? Will they not be delivered as a package for 2003 OS

[Bart_Heungens]

Hi,

There will be (indeed) no more PSP's released since the PSP and Firmware DVD are now bundled in the new SPP Service Pack for Proliant.

SPP's are only there for Windows 2008 and higher, so the PSP 8.70 is the latest one for Windows 2003...

Individual components will still be released if necesarry (bug fixes...) So keep checking the HP site...

Kr,

Bart

[Goutham_Sabala]

Thanks Bart , that is what I found out as well from here

http://h18004.www1.hp.com/products/servers/proliantessentials/foundationpack/index.html

HP Service Pack for ProLiant (SPP) is the the replacement product for Insight Foundation (PSPs, Smart Update Firmware DVD, and other systems software) and will be available at the following link: 

http://h18004.www1.hp.com/products/servers/management/spp/index.html

Insight Foundation(PSPs, Smart Update Firmware DVD, and other systems software) will be base lined at v 8.70 which means that it will continue to be shipped with HP ProLiant Generation 7 and earlier servers but will no longer be updated with new support. "Baselining" a product means that current formats of these products are not required moving forward. The Media Kit will be available on supported servers through the end of life of those servers and 5 years beyond.

[haven564]

CHIF drivers and new firmware released today for ILO3

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=4154847&prodTypeId=18964&prodSeriesId=4154735&swLang=8&taskId=135&swEnvOID=4064

[terryjj1]

Hi,

I see lots of comments and a note about the latest ILO 3 driver but has anyone tried the drivers and found it to be a solution to this long term problem?

thanks

Terry