BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.

[AdrianD]

Am also experiencing this issue, but have not yet raised a case. I will obtain the dump file from our Windows host so see if there is an issue. Am running 1.80 iLo2 on Prolaint DL380 G5 / 458567-421.

Informational iLO 2 03/11/2010 05:00 03/11/2010 05:00 1 BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.
Informational iLO 2 03/10/2010 17:56 03/10/2010 17:56 1 Server power restored.
Informational iLO 2 03/10/2010 17:56 03/10/2010 17:56 1 Server power removed.
Informational iLO 2 03/10/2010 17:56 03/10/2010 17:56 1 BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.
Informational iLO 2 02/21/2010 00:30 02/21/2010 00:30 1 Server power restored.
Informational iLO 2 02/21/2010 00:30 02/21/2010 00:30 1 Server power removed.
Informational iLO 2 02/21/2010 00:30 02/21/2010 00:30 1 BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.
Informational iLO 2 02/20/2010 14:23 02/20/2010 14:23 1 Server power restored.
Informational iLO 2 02/20/2010 14:23 02/20/2010 14:23 1 Server power removed.
Informational iLO 2 02/20/2010 14:23 02/20/2010 14:23 1 BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.
Informational iLO 2 02/13/2010 16:20 02/13/2010 16:20 1 Server power restored.
Caution iLO 2 02/13/2010 16:20 02/13/2010 16:20 1 Server reset.
Informational iLO 2 02/13/2010 14:11 02/13/2010 14:11 1 Server power restored.
Caution iLO 2 02/13/2010 14:11 02/13/2010 14:11 1 Server reset.

[Erdogan Temur]

Hi Adrian,

OS version ?

SOLUTION:
Advise clients to upgrade the following:
1. BIOS Last version Update
2. Controller card firmware update
3. iLO2 controller firmware to v1.81

This is how we thank each other in the forum

http://forums11.itrc.hp.com/service/forums/helptips.do?#33

Regards.

[Mike Blaszczak]

Which version of the two iLO Windows drivers do you have, Adrian? You'll need to check in the Windows Device manager.

[Steve McNutt]

I've done 1,2, and 3, but am having the same issue. In fact, I didn't have the issue until I did 1,2, and 3.

I update the firmware on my servers during the weekend of 2/20/2010. About a week later a server rebooted with these entries in the ilo log:
Informational iLO 2 02/25/2010 06:34 02/25/2010 06:34 1 Server power restored.
Informational iLO 2 02/25/2010 06:33 02/25/2010 06:33 1 Server power removed.
Informational iLO 2 02/25/2010 06:33 02/25/2010 06:33 1 BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.

Then this past weekend on a different server I got the same thing:
Informational iLO 2 03/13/2010 02:50 03/13/2010 02:50 1 Server power restored.
Informational iLO 2 03/13/2010 02:50 03/13/2010 02:50 1 Server power removed.
Informational iLO 2 03/13/2010 02:50 03/13/2010 02:50 1 BMC IPMI Watchdog Timer Timeout: Action=System Power Reset.

Needless to say, the power to both racks (actually two different sites) was fine, although thus far I've only gotten in once per server (though I have a couple other servers that I'm now sweating over).

Insult on injury, now when I try to logon to the second server it locks up the whole server and I have to reboot it (though the services on it remain operable so long as I don't logon), but I'll probably have to call Microsoft on that.

[Mike Blaszczak]

When we had this problem, we discovered it was because both iLO drivers needed to be updated. One didn't show up in Device Manager, and was out of date.

[8i5]

We've had this issue on our 64 bit BL465c servers for some time. We've updated all drivers to the latest version but we still get it from time to time without explanation. HP just keep saying update firmware and unfortunately another HP ilo management controller driver came out a month ago...1.13

No fix for this issue described in the fixes list for this driver.

[Reece Wilkinson]

Here's the advisory that explains the issue and provides instructions on how to fix...

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?locale=en_US&objectID=c01802766

I had 2 servers reboot because of ASR today and one of them was only ILO (not ILO2) so it may be caused by the driver more than the firmware. I upgraded the drivers/firmware for ILO and ILO2 to 1.15.0.0 so we'll see how it goes.