topic Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70 in Server Management - Remote Server Management

BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Tue, 17 Mar 2009 10:30:14 GMT

After upgrading to the latest support pack (8.20A) with Online ROM Flash for windows x64 ver 1.70 ILO2 and management controller 1.9.0.0 Im getting hpqilo2 warning messages in my Windows System Event log like these:
-(Event ID 57)--
Failed GET SENSOR READING, sensor 9
NetFN 0x4, command 0x2D timed out
Maximum indicator count (1) exceeded
Duplicate thermal sensor instances: 1
Duplicate thermal sensor indices: 1
Duplicate thermal sensor instances: 2
----
and on and on....
They appear twice every day, on ALL three of my BL 460C blades. This did not happen at all before this upgrade.
Is there something wrong with the firmware? Is there something wrong with all my blades? Or is there anything I can do to prevent this from happening?
Systems are running Windows Server ver 2003 R2 x64 with all security patches installed!
(Same firmware and controller driver works fine on my "older" ML370 systems!)

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

James Kennedy_4 — Thu, 19 Mar 2009 10:35:11 GMT

I am having these messages as well on my BL460c blades.

I've had the ILo 1.70 firmware on for a long time and hasn't been a problem until I installed the PSP 8.20 yesterday. So definately something with the drivers int he 8.20 pack that is doing it.

Also getting some blue screen crashes now too.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Thu, 19 Mar 2009 10:37:52 GMT

I'va also had a few BSOD crashes lately. Never happened before! So 8.20 is definately to blame. What to do? Go back to 8.15? Is that possible?

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

James Kennedy_4 — Thu, 19 Mar 2009 11:31:40 GMT

So far, it seems just specific to x64 machines. I'm hoping to not have to uninstall the entire PSP. I'd like to try to isolate which driver is causing it.

Right now I have ASR disabled on some of the machines, hoping to catch the crash memory dump this time.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Thu, 19 Mar 2009 12:50:44 GMT

It's only my Blade servers doing this. I have ML370's with 64 bits installation, and they are working fine with support pack 8.20. So I guess it's a BL460c/x64 issue? (I have no BL460c's with 32bit OS to compare with).

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

James Kennedy_4 — Thu, 19 Mar 2009 13:36:56 GMT

Yeah, just BL460c with x64.

I reverted the ilo2 management driver back to 1.8 and the hpqilo2 messages have not returned yet.

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=3709945&prodSeriesId=3808910&prodNameId=3808911&swEnvOID=1113&swLang=13&mode=2&taskId=135&swItem=MTX-03c99c6a52a047d0bf38df0331

Still waiting to see if there's any more crashes though.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Thu, 19 Mar 2009 15:53:38 GMT

I've now done the same here; downgraded to 1.8 on ONE of my blades. I'll keep an eye on it and see what happens. This hpqilo2 message only appeared twice a day here, so I may be in for an overnight wait. :-)

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Fri, 20 Mar 2009 12:29:52 GMT

Seems to me now that going back to the ilo2 management driver back 1.8 may have solved the problem. Since I did that on a few of the servers here, the problem seems to be gone. I will now downgrade ALL my servers here to the previous version (i DID get the same error messages on the ML370's as well eventually!) tonight, and keep an eye on them over the weekend.
Now what? Will HP do something about this, or what?

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

James Kennedy_4 — Fri, 20 Mar 2009 13:02:22 GMT

Yeah, so far so good here. It definately stopped the error message from appearing, and have not yet had any crashes. Time will tell.

PSP 8.20 isn't "officially" available until the 31st of the month. It was listed as available for servers for a few days, and then it was pulled. Perhaps HP discovered the issue and is working on a newer version of that driver.

I'll update again if I hear anything or have any further issues.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Fri, 20 Mar 2009 15:08:35 GMT

Well.... bad news... the error message reappeared here on one of the servers, so there is definitely more to it than just this driver. The error message is obviously not so frequent as it was before, but still there. The fact that HP has pulled the 8.20 update package I guess means they have realized something is wrong. I hope I can live with the problem until they re-release it?

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Cliff McQuigg — Tue, 24 Mar 2009 14:02:22 GMT

We're getting these too, as well as a LOT of blue screen crashes. Kind of a problem since we're rebuilding a lot of stuff to 64bit right now and our firmware/drivers are the most current.

Anyone find which driver or firmware is at fault?

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

James Kennedy_4 — Tue, 24 Mar 2009 14:05:23 GMT

For me, me upgrading to PSP 8.20 and keeping the downgraded version of the ilo2 management driver back at 1.8, solves all my issues so far.

No more errors, and no reboots... so far.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Tue, 24 Mar 2009 15:35:08 GMT

All my 64 bit systems seem to be stable now using ilo2 management driver 1.8 ! However two of my older 32 bit systems (with only ilo1) has developed BSOD's with no event error messages after I upgraded them with support pack 8.20. Non of my servers (neither ilo2 nor ilo1, 32 or 64 bits) has BSOD'ed before.
They were all stable with support pack 8.15, so my advice is to stay with 8.15B until HP has re-released 8.20.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Torbjorn Mohn — Tue, 31 Mar 2009 07:13:26 GMT

HP today re-released Support Pack 8.20(A): http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=no&prodTypeId=3709945&prodSeriesId=1842750&prodNameId=3288156&swEnvOID=1113&swLang=8&mode=2&taskId=135&swItem=MTX-7e8dfb45569e4425bb197254d7
Have anyone here had the guts to try it out? The 1.9 ILO2 controller seems to me to be the same in the "new" Support Pack?

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

James Kennedy_4 — Tue, 31 Mar 2009 11:31:38 GMT

Yeah, i'm afraid of that driver. I'm not going to be putting on any blades for a while until it gets tested. I'm sure there will be a lot more people rolling out the 8.20 support pack, now that it's available again. Keep an eye on the forums.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Rastislav Isunin — Sat, 04 Apr 2009 17:59:03 GMT

The same issue on new DL 580 G5 with Windows 2003 x64.
After facing BSOD issues according recomendation implemented newest PSP - unfortunately 8.20(A). Right after installation server bluescreened on crappy Network driver bundled here. I have upgraded this particular driver to one released March 27 and server seems to be stable, but this hpqilo2 event Warnings ID 57 start comming to system log daily. So no improvment in (A) version done regarding this issue.

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Kostas Kakogiannis — Mon, 13 Apr 2009 13:19:26 GMT

Try installing,

HP ProLiant iLO 2 Management Controller Driver for Windows Server 2003
Version :1.11.0.0
7 Apr 2009

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=3709945&prodSeriesId=1842750&swItem=MTX-295127d6d21d4885a860038d5b&prodNameId=3288156&swEnvOID=1005&swLang=8&taskId=135&mode=4&idx=2

It seems to correct this issue...

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Kostas Kakogiannis — Mon, 13 Apr 2009 13:33:06 GMT

Sorry, the above link is for x86 win2k3, here is the link for Win2k3 x64!!!

HP ProLiant iLO 2 Management Controller Driver for Windows Server 2003 x64 Editions

Type: Driver - System Management
Version: 1.11.0.0 (7 Apr 2009)
Operating System(s): Microsoft Windows Server 2003 for 64-bit Extended Systems

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=3709945&prodSeriesId=1842750&swItem=MTX-b016a4092d95486b88c4ebe86d&prodNameId=3288156&swEnvOID=1113&swLang=8&taskId=135&mode=4&idx=2

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Rcash_1 — Tue, 14 Apr 2009 03:51:20 GMT

I installed Version: 1.11.0.0 on my blade bl460 server running Win2008 x64 and it did not help. It continues to report hpqilo2 errors in the event log then reboot (ASR). Went back to version 1.8 and so far the errors have not returned.

Installed version 1.11 on a ML350 G5 running win2008 x64 and it also started reporting hpqilo2 errors and reboots (ASR) went back to 1.90 on it and errors went away.

Installed version 1.11 on a DL360 G4 runnning Win2008 x86 and so far it has no errors and no reboots (ASR)

Re: BL460c ILO2 problem after firmware (ROM flash) upgrade to 1.70

Kostas Kakogiannis — Tue, 14 Apr 2009 05:33:36 GMT

My suggestion as a workaround, would be, in the event, you see errors in system event log like :

57 hpqilo2 Duplicate thermal sensor instances: 5
57 hpqilo2 Duplicate thermal sensor indices: 6

To try installing iLO2 Driver 1.11.0.0

If after installation of driver and possible OS restart errors persist, try versions 1.0.8.0 and 1.0.9.0 respectively.