ILO2 Disabled and Can't enable on DL380-G6 - anyone else?

[Jabkowski Matt]

On February 5th the ILO2 chips were disabled on five of my servers. We were not working on the servers. Two DL380-G6's, Two DL360-G5's and one ML370-G5 all have the same problem. THe problem is more obvious on the DL380-G6's becasue the cooling fans all went to hight speed. One of the tech support specialists tells me this is because all environemntal controls on the G6's have been moved to the ILO chip.

Is anyone else having this problem?

They are trying to change the motherboards but I can't believe 5 motherboards on different model servers and different OS's would all go bad at the same time.

[Jabkowski Matt]

And what is GCC?

The next day after this happend I was instructed to upload HPS Reports on all servers and upload them to an FTP site. It has been 3 days and I can't get an status update as to what if anything is being done with the reports or if anyone is even working on it. All I get is an email from a customer support person that the reports are uploaded, have been sent to GCC and they will keep me posted. But I never getting any updates.

[Wallace H Wyeth]

Yes we are having the same issue. For no apparent reason, the fans on all our G6 servers went to high speed last Friday, 4/23/2010.

Have you been able to resolve your issue?

[JPM79]

We began seeing this on all our G5s and G6s on 5/8. All apparently went offline at about the same time according to our logs, and is affecting Windows 2003/2008 (with event 57 showing up in the event logs) as well as vSphere 4 hosts. The problem appears to be limited to our native VLAN. Our other iLO 2 hosts at other sites are unaffected, so this leads me to believe that there is some sort of rogue network device causing the issue. If I shut down a given server, remove power completely, unplug iLO, and power it back on the server functions normally until I reconnect it to our native VLAN. iLO also functions if I cross-connect it directly to a laptop. Did you ever find a resolution to your issue? I have not contacted HP yet but plan to first thing next week. The 1.82 firmware has not helped.

[Wallace H Wyeth]

We are still working on this issue. HP has asked us to update the firmware on our switches which we plan to do later this week.

In the interim, we have connected the iLO ports of all our G6 servers to an older HP Procurve 2848 switch. We are not seeing the iLO lockup when connected to this older technology.

Please keep this thread updated if you learn more.

Wallace

[Wallace H Wyeth]

We have also applied the 1.82 update and as JPM stated it did not resolve the issue.

JPM I agree that you should log this with HP. Each additional report will help them learn more about the cause of this failure.

Can you post what switches you are using?

[JPM79]

Do you have a case # I can reference when calling?

I am seeing the issue when I connect an iLO 2 interface to any port configured for the native VLAN on our Cisco 6513 running IOS v 12.2(33)SXH2a or a Cisco 3548 runnign IOS v 12.0(5)WC3b. I did verify that if I place the iLO connection on another other than the native VLAN then the card continues to respond. However, as soon as I connect it to the native VLAN port the fans go high speed and it stops responding, even if it doesn't have a valid IP for the native VLAN. My next step will be to try placing a hub between the iLO and the switch.

I am wondering if spanning tree on the Cisco is coming in to play (we have everything set to portfast default and no ports are blocking, but I am wondering if the HP iLO 2 cards don't like the bpdus.)

[Wallace H Wyeth]

HP has created several cases but this seems to be the central case number.

CASE:4613617133

We do not have spanning tree enabled, but this a PoE switch. I wonder if PoE is in any way involved.

The most interesting thing is that most of these servers have been working fine for a long time. Nothing obvious changed at the time of failure. Clearly this is happening to a number of HP servers so what is the common factor?

[JPM79]

Thank you, hopefully this case # will convince the HP rep I am on hold with to assist me (they are trying to say that it is a network issue). I am having the issue on a non PoE switch (the Cisco 3500XL), but also observer it on the PoE line cards in our 6513.

[JPM79]

HP is saying that case is closed and that it was an issue the switch. No other details available.