BL20p G4 ILO issue

[raoul_schaffner]

I updated the ILO of a BL20p G4 blade to version 2.15 (current as of now for ILO2). Only afterwards did i learn, that while the 2.x series of ILO is installable via the Support Pack for Proliant, it is no longer supported on the p-series of blades.

So, right now the ILO is no longer pingable, nor does a manual press of the power button start the blade. The private ILO address 192.168.1.1 (via dongle / cable exploder) also does not ping.

Is there any way to boot a blade when the ILO is hosed? If so, I could downgrade ILO from within the booted operating system.

Thank you very much for any hints,

Raoul Schaffner.

[TVVJ]

iLO’s functionality includes enabling the server to power on. In case of p-Class Blades, the server’s iLO communicates with the power management module of the p-Class enclosure of the server’s power requirements and based on available power available in the enclosure allows the server to power on.

In the present situation this communication is not occurring as the firmware of the server’s iLO is not able to communicate with the enclosure’s power management module.

I am not able to see a solution other replacing the system board. Having said that, try connecting the Blade to a p-Class Diagnostic Station and check if it powers on. If not then perform the following steps, with the server connected to Diagnostic Station:

a.      Clear the server’s NVRAM. To do so, the following steps may be performed:

           i. Power down the server and disconnect it from the Diagnostic Station.

           ii. Please refer to item 14 of the System board components diagram, on page 56 of 66 of the server’s maintenance and service guide to identify the system maintenance switch. The guide is locatable at the following link:

           http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c00715963/c00715963.pdf

            iii. Locate individual switch S6, and set it from the default position of ‘Off’ to ‘On’. Replace the server in the bay and power it on. Wait till the message indicates that the server has been reset. Then power off the server, remove it from the bay, revert the switch S6 to position back to ‘Off’ and then power it on.

             After clearing NVRAM, and the server power’s on, please re-enter the serial number in the server's RBSU. You may perform the following steps:

             i. During the server startup sequence, press the F9 key to access RBSU.

             ii. Select the System Options menu.

             iii. Select Serial Number. The following warning is displayed:

             WARNING! WARNING! WARNING! The serial number is loaded into the system during the manufacturing process and should NOT be modified. This option should only be used by qualified service personnel. This value should always match the serial number sticker located on the chassis.

              iv. Press the Enter key to clear the warning.

              v. Enter the serial number and press the Enter key.

              vi. Select Product ID.

              vii. Enter the product ID and press the Enter key.

              viii. Press the Esc key to close the menu.

               ix. Press the Esc key to exit RBSU.

               x. Press the F10 key to confirm exiting RBSU. The server will automatically reboot.

b.      If the server is able to power on after clearing NVRAM, then attempt troubleshooting steps detailed in the solution section “Maintenance and Configuration Options > Recovering from a failed iLO 2 firmware update” contained in HP Integrated Lights-Out 2 (iLO 2) HP Guided Troubleshooting. This troubleshooting solution can be located at the following url:

http://h20584.www2.hp.com/hpgt/guides/select?lang=en&cc=us&prodTypeId=329290&prodSeriesId=1146658

c.       The iLO firmware latest supported on the server is1.70 (B), downloadable from the following link:

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=3709945&prodSeriesId=1121598&swItem=MTX-613c4a32c1ec4192b0e9b5fd41&prodNameId=3288176&swEnvOID=4024&swLang=8&taskId=135&mode=4&idx=1

Please review steps to perform the update under Tab Installation Instructions.

d.      If the server does not power on when connected to the Diagnostic Station, then it would mean the system board may require replacement. Contact HP support with the server’s serial number:

http://h20565.www2.hp.com/portal/site/hpsc/public/help/contactHP/home/

Regards,

Vijayasarathy

[Oscar A. Perez · Accepted solution]

Since p-Class blades are no longer supported in iLO2 versions 2.xx, we did not test iLO2 on the BL20p G4 and did not see that with firmware version 2.12 and 2.15, iLO2 is crashing on boot on these systems.

There is a way to restore iLO2 using the following steps:

EDIT:  Use HP Smart Update Firmware DVD version 10.10 instead of Firmware CD 8.70 or Firmware DVD 9.30

Download the Use HP Smart Update Firmware DVD version 10.10. The ISO image is available for download from the following URL:

http://bizsupport1.austin.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?swItem=MTX-9ed665a89aba447d925937f38b

Download the iLO 2 firmware v1.82 Component for Linux (CP012567.scexe) from the following URL:

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?swItem=MTX-b1c7bb6f42054394b8012695e9

Download the HP USB Key Utility for Windows and create a bootable USB Flash drive using  HP Smart Update Firmware DVD version 10.10.
http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?swItem=MTX-40e268a99cc243a299352b7b20

Copy the CP012567.scexe to the USB Key under hp/swpackages folder.

Set the iLO 2 Security Override Switch on the system board to the ON position. The location of the switch is printed on a label located on the inside of the hood cover.

Connect the p-Class dongle to the front of the p-Class blade server, attach the USB key, keyboard and monitor and Power ON the p-Class server. Note that because iLO2 isn't responding, the server could take up to 30 minutes to show POST.

Once the server boots from the USB key, select Interactive mode and wait until the Select Language screen shows up.

Use the following key strokes to exit out of HP Smart Update Manager. A command prompt will be displayed over the HP Smart Update Manager window. The command prompt takes approximately 30 seconds to appear:
CTRL + ALT + d + b + x (Keep the CTRL and ALT keys pressed when typing d b x )

At the command prompt, navigate to the HP Smart Update Firmware DVD directory containing the supplemental iLO 2 firmware update by using the following command:

  cd mnt/cdrom/hp/swpackages

Use the following command to unload the HPILO module:

  rmmod hpilo

Use the following command to execute the iLO 2 firmware update in direct mode:

  sh CP012567.scexe --direct (This parameter requires two dash (-) characters).

Once the iLO 2 firmware upgrade is completed, power OFF the server and set the iLO 2 Security Override Switch on the system board to the OFF position.

[raoul_schaffner]

Hi all,

thank you very much Vijay and Oscar for your valuable hints.

However, I did not yet manage to get into the command promt mode of the Firmware CD 8.70

1) I can boot the blade with the security override switch #1 -- OK

2) I can boot from the 8.70 USB Boot Stick or the 8.70 FW CD-R (CD drive also via USB) -- OK

3) there is no interactive mode to select, just "Boot into firmware CD 8.70" -- it seems to be interactive by default

4) I tried the key combination ctrl + alt + d + b + x (pressed all at once, or holding ctrl + alt and typing d b x, tried both) at several stages of the following process:

- booting

- loading drivers

- starting Application (Title: Proliant Firmware Maintenance CD)

- select Language (here I had to unplug the Keyboard to use the Mouse)

- License Agreement

- Home

- Tab "Firmware Update"

- "Install Firmware" (here "HP Smart Update Manager" gets loaded)

5) after loading HP SUM, I switched back Mouse and Keyboard and again tried the key combo, to no avail.

--> Is it confirmed that this works with version 8.70 of the Firmware Maintenance CD? I have read elsewhere of this key combo, but in combination with FW CD 9.30...

Greetings and many thanks again,

Raoul Schaffner.

[Oscar A. Perez]

Let me test this on my Bl20p G4 real quick and I will post the results.

[Oscar A. Perez]

Ok, the key combination Ctrl + Alt + d + b + x does NOT work in Firmware CD 8.70 for some reason but, it does work in Firmware DVD 9.30 so, I edited the recovery procedure above to use DVD 9.30 instead.

Please notice that with Firmware DVD, the location of swpackages folder changed and I'm now using CP012567.scexe which is iLO2 version 1.82

Let me know the results.
Oscar

[raoul_schaffner]

OK, here's the newest status:

The blade without ILO

1) can boot FW CD 8.70

2) cannot boot FW CD 9.10C (stalling at blue progress bar)

3) cannot boot FW CD 9.30 (stalling at blue progress bar)

Somehow, the missing ILO is affecting the boot process of 9.10C and 9.30

Just out of curiosity, I tried FW CD 9.30 on a different blade with working ILO. This boots fine, and I can confirm the key combo is working at the "Select Language" screen.

So my next step is to test FW CD 9.00 and 9.10B... my only hope, it seems. :-/

Cheers and thanks for any other hints,

Raoul.

[Oscar A. Perez]

It seems Firmware DVD won't start if iLO isn't responding.

Can you just boot the server to Linux?

All we need is to execute the below command while the iLO Security Switch is ON

sh CP012567.scexe --direct

[Jimmy Vance]

or boot into the HP Scripting toolkit shell. Same kernel the other utility images are based from.  Download and run the script to make a bootable image and then copy the required .scexe file to the image

older 8.70 toolkit

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdDetails/?javax.portlet.endCacheTok=com.vignette.cachetoken&javax.portlet.prp_bd9b6997fbc7fc515f4cf4626f5c8d01=wsrp-navigationalState%3Didx%3D%7CswItem%3DMTX_2387cbb29fd245509e5e171001%7CitemLocale%3D%7Cmode%3D%7Caction%3DdriverDocument&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.tpst=bd9b6997fbc7fc515f4cf462... 

latest version 9.40

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdDetails/?javax.portlet.endCacheTok=com.vignette.cachetoken&javax.portlet.prp_bd9b6997fbc7fc515f4cf4626f5c8d01=wsrp-navigationalState%3Didx%3D%7CswItem%3DMTX_482e23f4092048faadf41a71bc%7CitemLocale%3D%7Cmode%3D%7Caction%3DdriverDocument&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.tpst=bd9b6997fbc7fc515f4cf462...

[Oscar A. Perez]

I just tested HP Smart Update Firmware DVD version 10.10 and it works fine. I updated the recovery procedure once again.