after update BMC firmware DL120 G6 don't start

[Andrey13]

Hi All!!

 I have a  DL 120 G6 server. 

I update  bios firmware to version 0260701 (07/01/2013) - all work fine

Then I update BMC firmware to version 4.26 ...after that server don't start..

Situation very similar with this:

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay?javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken&javax.portlet.prp_ba847bafb2a2d782fcbb0710b053ce01=wsrp-navigationalState%3DdocId%253Demr_na-c02865750-1%257CdocLocale%253D%257CcalledBy%253D&javax.portlet.tpst=ba847bafb2a2d782fcbb0710b053ce01&sp4ts.oid=3984621&ac.admitte...

..but bios newer...

Thanks in advance

p.s. Sorry for my english :(

[Torsten.]

Do you get the same fan related message?

[Andrey13]

Yes, "Fan fail/missing. System prepare to shutdown"

[Andrey13]

I can login into LO from network...

in system log :

........

Generic 12/17/2013 13:17:26 CPU FAN1 Upper Non-critical-going high Deassertion
Generic 12/17/2013 13:17:27 CPU FAN5 Upper Non-critical-going high Assertion
Generic 12/17/2013 13:17:27 CPU FAN5 Upper Critical-going high Assertion
Generic 12/17/2013 13:17:28 CPU FAN1 Upper Non-critical-going high Assertion
Generic 12/17/2013 13:17:28 CPU FAN1 Upper Critical-going high Assertion
Generic 12/17/2013 13:17:28 CPU FAN5 Upper Critical-going high Deassertion
Generic 12/17/2013 13:17:28 CPU FAN5 Upper Non-critical-going high Deassertion
Generic 12/17/2013 13:17:28 SYS FAN8 Upper Critical-going high Deassertion
Generic 12/17/2013 13:17:28 SYS FAN8 Upper Non-critical-going high Deassertion
Generic 12/17/2013 13:17:29 CPU FAN1 Upper Critical-going high Deassertion
Generic 12/17/2013 13:17:29 CPU FAN1 Upper Non-critical-going high Deassertion
Generic 12/17/2013 13:17:29 CPU FAN2 Upper Critical-going high Deassertion
Generic 12/17/2013 13:17:29 CPU FAN2 Upper Non-critical-going high Deassertion
Generic 12/17/2013 13:17:29 CPU FAN3 Upper Critical-going high Deassertion
Generic 12/17/2013 13:17:29 CPU FAN3 Upper Non-critical-going high Deassertion

................

in sensor monitoring:

...

Voltage System 12V Unavailable
Voltage System 5V Unavailable
Voltage Standby 5V Normal operating range 5.1714 Volts
Voltage System 3.3V Unavailable
Voltage Standby 3.3V Normal operating range 3.348 Volts
Voltage CPU0 Vcore Unavailable
Voltage System VTT Unavailable
Voltage DDR3 1.5V Unavailable
Button Identify State Deasserted 0
Button NMI State Deasserted 0
Temperature PCI Ambient Temp Normal operating range 21.5 deg. C
Temperature DIMMA1 Temp Normal operating range 25.5 deg. C
Temperature DIMMA2 Temp Unavailable
Temperature DIMMA3 Temp Unavailable
Temperature DIMMB1 Temp Unavailable
Temperature DIMMB2 Temp Unavailable
Temperature DIMMB3 Temp Unavailable
Temperature CPU0 Dmn0 Temp Normal operating range 30 deg. C
Temperature CPU0 Dmn1 Temp Unavailable
Temperature DDR Ambient Temp Normal operating range 25 deg. C
Temperature SYS Ambient Temp Normal operating range 24 deg. C
Processor CPU Prochot Limit Not Exceeded 0
Processor CPU0 CATERR State Deasserted 0
Processor Therm-Trip0 State Deasserted 0
Fan CPU FAN1 Unavailable
Fan CPU FAN2 Unavailable
Fan CPU FAN3 Unavailable
Fan CPU FAN4 Unavailable
Fan CPU FAN5 Unavailable
Fan CPU FAN6 Unavailable
Fan SYS FAN7 Unavailable
Fan SYS FAN8 Unavailable

....

Very mach sensors with status "Unavailable"

[Andrey13]

I try to recovery BMC..in accordance with this :

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay/?sp4ts.oid=3884339&spf_p.tpst=kbDocDisplay&spf_p.prp_kbDocDisplay=wsrp-navigationalState%3DdocId%253Demr_na-c01995737-3%257CdocLocale%253D%257CcalledBy%253D&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken

But after set jumper 27 to state "Recovery BMC" keybord and mouse did not work!!! and I can't reupdate BMC firmware..

[Andrey13]

I try to use usb keyboard and procedure BMC recovery complete succesfully!

...but..I see again "Fan fail\missing. System prepare to shutdown"

[Andrey13]

...And the violence caused such silence....

problem solved :

1. Update firware BMC initial version (4.22) (set jumper BMC recovery)

2. update bios previous version (need also set jumper BMC recovery, because system coud't start)

!!! it works!!!

3. update firware BMC latest version 4.26.

4. Update bios latest version (O26 (07/01/2013)

!!it works too!!!

thanks for attention.

p.s. кароч хз че было...

[Genai]

Had same problem.
Solved - thank you guys!

Probably,
if update first BMC, then BIOS everytihing ok.
if update BOIS, then BMC something doesn't work.

p.s. просто глюк )

[Vincenthp]

Genai do you have the previous BMC firmware ?

I cannot download it from HP without paying. Very annoying and left me with a useless server. 

Thanks in advance

Vincent 

[adriandanciu]

Hello.

Do you have the old version of bios usb key file?

If you have can you give me please?

Thank you.

[drookie]

So... one hour of hardcore after I stepped onto this issue.

I'm replying here just to summarize some approaches and clear things.

Yes, if the BIOS is updated to 26 version first and the lo100i firmware is updated next, the issue is triggered.

The server can be forced to boot without BMC recovery mode, using this cruel hack(I had to experiment, 'cause it's in remote datacenter): power it up from ilo, then, as soon as you'll get the BIG WHITE POST screen, issue a "Hard reset" from ilo. Experiment with timings when you issue it, - it's not happening every time.

However, I don't see the way of resolving this without a BMC recovery mode.

[HPHash]

Vincent,

I have similar issue. Do you have the old BMC and BIOS? 

I tried re-flash with latest BMC and BIOS, but no luck.

Let me know if you have working solution.

Regards,

Hash

[Dimkka82]

Hello!
Could you send me the old bios and bmc files?

[Alex-serv]

The same issue, but unable to download any previous bios version.

Could anyone help me out with this?

[tdtorn]

I've faced the same issue with HP DL160G6. Cannot get over it after 4 months and 2 servers are down... ((  Do you have old BIOS? Can you share it, please?

[tdtorn]

Your link won't open, but I found another doc with this discription. But it doen's specify the location of this jumber. Anyone of these jumpers (see the link or picture) ? 

https://dunlopcis-my.sharepoint.com/:i:/g/personal/a_toropchin_dunlopcis_ru/EfdcUe1Pl1lMuu48amzhwdEBlSrqwYz2yDrUjUmU-44elg?e=w8Uu3q

[tdtorn]

Hi, Vincent! Sorry for bothering with this old issues... I have the same problem with 2 servers and really need old BIOS version. Did you manage to find it? Is it posbile to get it ... for others? )

[JoelR01]

Thanks to Andrey13 whose post helped me a lot. This took me several hours, so hopefully I can help someone else with this issue.

It turns out all of the files are available without an active service contract, but you have to get them from the following specific downloads I was able to find on the DL120 G6 product downloads page.

I'd repackage them, but it'd probably get removed. So here's what to get:
SP52417.exe: Firmware Upgrade for HP ProLiant G6/G7 Lights-Out 100 Remote Management (For USB Key-Media) -> Version: 4.22 (A) Apr 7, 2011
SP62696.exe: Firmware Upgrade for HP ProLiant G6/G7 Lights-Out 100 Remote Management (For USB Key-Media) -> Version: 4.26 (A) Jul 26, 2013
SP62832.exe: System ROMPaq Firmware Upgrade for HP ProLiant DL120 G6 Servers (For USB Key-Media) -> Version: 2013.07.01(A) Sep 13, 2013
cp015398.exe: Smart Update Firmware DVD firmware-10.10-0.zip\FW1010.2012_0530.49.iso\hp\swpackages\ -> Version: 10.10 Jun 4, 2012

Prepare a folder structure:

bios_2011.04.14\CPQO2602.68E (from cp015398.exe)
bios_2011.04.14\CPQSRVR2.CPU (from SP62832.exe)
bios_2011.04.14\ROMPAQ.EXE (from SP62832.exe)
bios_2013.07.01\CPQO2602.AE1 (from SP62832.exe)
bios_2013.07.01\CPQSRVR2.CPU (from SP62832.exe)
bios_2013.07.01\ROMPAQ.EXE (from SP62832.exe)
ilo_4.22\CPQQ1402.647 (from SP52417.exe)
ilo_4.22\HPRPQBMC.CPU (from SP52417.exe)
ilo_4.22\ROMPAQ.EXE (from SP52417.exe)
ilo_4.26\CPQQ1402.AA8 (from SP62696.exe)
ilo_4.26\HPRPQBMC.CPU (from SP52417.exe)
ilo_4.26\ROMPAQ.EXE (from SP52417.exe)

- Run Rufus, flash FreeDOS onto a USB flash drive
- Copy 4 folders onto USB flash drive. Insert into DL120.
- Set BMC PASSWD RESET/RECOVERY jumper to the 2 pins closest to the IO side of the motherboard (rear of the case)
- Leave all other DIP switches to original position
- Unplug other drives so you don't need to set the boot order after each reboot/power cycle. Optionally, remove all but one DIMM so the full memory test doesn't take forever between reboots.
- Boot to FreeDOS, go to ilo_4.22 and run rompaq
- Shut down, change BMC jumper to original position
- Start up, let the BMC fw downgrade complete
- Shut down, set BMC jumper back to recovery
- Boot back into FreeDOS, go to bios_2011.04.14 and run rompaq
- Reboot back into FreeDOS, go to ilo_4.26 and run rompaq
- Shut down, change BMC jumper to original position
- Start up, let the BMC fw upgrade complete
- Boot into FreeDOS, go to bios_2013.07.01 and run rompaq

Now you can configure your BIOS, and everything should be good.

Edit: It turns out I had moved around the BMC recovery jumpers beforehand and lost track of how it was set up when I was doing this. Additionally, my BIOS battery was depleted. So you could probably leave the BMC recovery jumper closest to the edge of the motherboard to the reset position and ignore the frequent swapping of its position until you're all done.