BL460c's hanging up

[Eric_76]

"Also looking for solution"
we to are having the same exact problems.
we have multiple blades dropping off and OA say it is OK and green but physically the server has an amber light in it and nonresponsive the only way to get it back is to un-plug and re-plug in the blade.
we have BL460g1 ilo 1.50 w2k3 std.
BladeSystem c7000 Enclosure ver. 2.25

[Ian McGann]

Wow, I had totally forgotten about this thread. Has the problem gone away? Pretty much.. Do I know why? Not exactly...

In short, I battled back and forth with a local CE and HP Escalation on this. Upgrading to iLO 1.60 seems to have fixed the problem, however we did have one server with the same problem after the 1.60 update.

So, it seems that iLO 1.60 fixed the problem, but without explanation of why the one server froze with it.. HP has been unable to give me a reason other than "It seems as if the issue has been resolved."

[Dan Silva]

Hi, everyone:

Based on my experience, it appears iLO Firmware 1.60 should fix this problem even though it is not mentioned in the "fixes" page.

Download Drivers and Software: http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?lang=en&cc=us&prodNameId=3288156&taskId=135&prodTypeId=3709945&prodSeriesId=1842750〈=en&cc=us

Choose your OS, then "Firmware - Lights Out Management".

If anyone has this hang after updating to 1.60 (or recently released 1.61), please post to this group and log a case with HP.

I do not believe a system board will fix the issue, so if you have a lot of servers that have this same issue (especially if they're in different enclosures), it's important you log a case with HP so it can be analyzed.

But, of course, try the iLO firmware update first and update the System ROM, ProLiant Support Pack, etc, to make the call center happy.

Regards,

Dan

[Ian McGann]

Thanks again, Dan.

Ian

[Fabio_S]

Hello,
same problem here. I first tried and update firmwares on blades ILO's and enclosure's OA, after reading here, a couple of months ago.

But the issue appeared again yesterday, trying to reboot one of the blades.
It seems like it can't start and hangs there in grey screen. Nothing works but physically unplugging and re-plugging the blade.

My feeling is about something hw-related in the connection between enclosure and blades, since this happens on all blades (not always).

Anyway, I've just updated enclosure's OA and only the involved blade's ILO to a new fw release.

If the issue persists, I'll open a ticket with HP.

I'll let you know....

[Ken Henault]

I have seen many strange issues like this fixed by reseating the OA tray. Not just the OAs, but the tray they sit in. This will require that the OAs are removed first, then the tray, then reseat everything. This can be done without disrupting the servers.

I've seen this fix so many strange issues that I now make it part of my install to reseat every OA tray during the install. They just seem to have an issue when in transit. Once reseated the issues don't come back.

I hope this helps with this issue.

Ken

[rvilat]

Hi,

We have the same problem here, running BL460c's in C7000 chassis. The servers are 2xE5430's, 32Gb ram running suse linux 10SP1.

When rebooting the servers, the server will shutdown ok, but does not reset or POST. ILO shows a blank screen and the VSP does not connect. Interestingly, the OA shows the power consumption as 509W compared to 200W when the blade runs normally. The problem is intermittent and we have had 8 out of 72 blades across 5 chassis fail so far.

Reseating the blade has not always worked, usually a HP engineer has come onsite and removed RAM, mezzanine cards, CPU's etc until it comes back to life.

HP's standard response has been to upgrade ILO/BIOS/OA/etc but the problem exists at all firmware versions. I do not think it has been addressed by HP.

Has anyone found a satisfactory solution to this?

thanks
rosh

[Simon Grant IRL]

I have had the same problem with 2 bl460c's. I rebooted one of them yesterday and it failed to reboot. No video via the port on the front and just the grey screen via ILO. As mentioned above no temperature data is reported for that blade in the OA console. I rebooted a 2nd today and had the same issue.

I have 5 BL460c's running ESX, all at ILO v1.5. OA is at v2.25

Has anyone gotten any further with the root causes? I am afraid to do anything with the remaining blades. Thanks to HA / VMotion all my VM's are still up and running, just on fewer hosts.

[Ian McGann]

99% of our issues stopped after upgraded the servers to iLO v 1.60

[rvilat]

We were running iLO 1.70, OA 2.32 and BIOS 01/APR/08 when we had the problems.

Just upgraded to OA 2.41 and BIOS 02/NOV/08 and are doing a reboot test every 1hr to see if the problem can be replicated.