BladeSystem - General
1748243 Members
4314 Online
108760 Solutions
New Discussion

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

 
SOLVED
Go to solution
drolfe
Valued Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

HI,

 

Bay Manufacturer Part Number Spare Part Number Firmware Version
1 HP 456204-B21 503826-001 4.21 Apr 12 2014
2 HP 456204-B21 503826-001 4.21 Apr 12 2014


Bay Device Model Firmware Version
1 HP VC Flex-10 Enet Module 4.20
2 HP VC Flex-10 Enet Module 4.20
3 3Gb SAS Switch 2.2.17.0
4 3Gb SAS Switch 2.2.17.0

 

 

I have one chassis fully updated to the latest SPP or newer, OA 4.21, VC 4.20 iLo 2.25, I"m still seeing the issue: please see below from the OA logs, not I've not been logged into the OA at all dince these events have been happening:

 

May 4 03:16:32 OA: Blade 8 found a partner device.
May 4 03:16:32 OA: Blade in bay 8 has been powered on

 

May 4 12:34:04 OA: Management Processor on Blade 9 appears unresponsive.
May 4 12:35:04 OA: Blade 9 has been allocated a default power value of 509W because iLO appears unresponsive.

 

May 7 12:41:40 OA: Management Processor on Blade 4 appears unresponsive.
May 7 12:43:00 OA: Blade 4 has been allocated a default power value of 509W because iLO appears unresponsive.

 

May 8 08:27:55 OA: Blade 4 found a partner device.

May 8 08:27:55 OA: Blade in bay 4 has been powered on

 

May 9 04:25:19 OA: Management Processor on Blade 7 appears unresponsive.
May 9 04:25:34 OA: Blade 7 has been allocated a default power value of 509W because iLO appears unresponsive.

 

I have no idea what could be causing these issues, Ilo network is on an isolated vlan from the rest of my network.

 

Please help

 

Regards, Daniel

drolfe
Valued Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Thought I would reply to say this is still going on.

 

Please see below what I've extracted from the OA system log
 
Please see below what I've extracted from the OA system log

Jul 15 20:29:22  OA: Management Processor on Blade 3 appears unresponsive.
Jul 15 20:29:42  OA: Blade 3 has been allocated a default power value of 509W because iLO appears unresponsive.

Jul 18 22:50:11  OA: Management Processor on Blade 9 appears unresponsive.

Jul 26 05:56:24  OA: Management Processor on Blade 3 appears unresponsive.

Jul 27 05:27:42  OA: Management Processor on Blade 5 appears unresponsive.
Jul 27 05:28:02  OA: Blade 5 has been allocated a default power value of 509W because iLO appears unresponsive.

Jul 27 06:34:37  OA: Management Processor on Blade 6 appears unresponsive.

Jul 28 08:12:48  OA: Management Processor on Blade 9 appears unresponsive.
Jul 28 08:14:28  OA: Blade 9 has been allocated a default power value of 509W because iLO appears unresponsive.

Jul 28 12:13:47  OA: Management Processor on Blade 8 appears unresponsive.

Aug 12 07:03:48  OA: Management Processor on Blade 9 appears unresponsive.
Aug 12 07:05:28  OA: Blade 9 has been allocated a default power value of 509W because iLO appears unresponsive.

 again, the only way to recover these devices is with an E-Fuse reset

 

HP replaced one of the G6 server boards and I haven't seen the issue again

 

ilo was recommended to be updated to 2.20 but the Server Board bios is still I24 05/05/2011

 
Do we think these ilo2 errors were caused but faulty firmware toasting the motherboards ?
 
Looking at this post, he seems to have the same issues as me:
 
 
 
All my current g6 servers where working without issues for years, once I shipped them into the New DC I upgraded all the blades to 2013-2014 firmware and these issues started to pop up. ??
 

 

Oscar A. Perez
Honored Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

When the iLO2 is showing the error, can you communicate to iLO2 from the OS?
Example, sending XML scripts to iLO2 via hponcfg from the OS?



__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
drolfe
Valued Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Hi,

No it's totally dead from the OS also, I can't communicate via hponcfg

 

I also can't access the ilo web interface, or ping the ILO ip address from either the network or from the OA itself

Regards Daniel

Oscar A. Perez
Honored Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

I'm assuming these iLO2s were working fine in the past. Do you know what changed in your environment that could have contributed for this issue to surface?

I sent you a PM.



__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
drolfe
Valued Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Hi,

 

Yes I replied to your PM :-)

 

If you have some BETA firmware for me to test that would be great!!

 

Regards, Daniel

drolfe
Valued Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Hi,

 

I'm currently testing Beta iLo2 2.26 firmware with the "Watchdog" proccess runnning

 

This "Watchdog" is to check on the hour if iLO2 is ok and if not locally reset the iLo2 ROM locally.

 

This saves me from having to E-Fuse reset blade servers to regain access to iLo.

 

So far so good, it's been around 5 days and I haven't had anymore blades stuck in iLo error but I'll need atleast 2-3 weeks to know for sure.

 

On another note, I can see this "Watchdog" proccess is enabled by defauilt on iLo3 (Bl460c G7) 

 

iLO3_auto_watchdog_restart2.jpg

 

So I guess the question is. Why isn't the watchdog enabled by default on iLo2 ?

 

Regards, Daniel

drolfe
Valued Contributor
Solution

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Hi,

 

This issue has been resolved

 

I'd like to thank Oscar A. Perez for taking the time to provide me with some BETA iLO2 2.26 firmware with the watchdog enabled.

 

Please note, this is BETA firmware and is likely NOT supported by HP so please use at your own risk

 

iLo 2 Firmware 2.26 watchdog enabled BETA

 

Regards, Daniel

Server-Support
Super Advisor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Dan,

 

Did you finally update the VC firmware or not needed for your case?

Best regards,
drolfe
Valued Contributor

Re: C7000 OA4.01 BL460c G6 iLO 2 version 2.22 dropouts

Hi,

 

No only the ilo firmware was needed.

 

Regards, Daniel