1500 iLO 4 Configuration is temporarily unavailable (5)

[NJK-Work]

We upgraded the iLO 4 firmware on a DL360p Gen8 with no errors or problems.  20 minutes later the server shutdown.  After booting back up, we get these errors at post:

iLO 4 not responding.  System Health Monitoring has been disabled

1500 iLO 4 configuration is temporarily unavailable (5).

After some analysis, the server appears to overheat and shuts down.  The top of the server is scalding hot.  My guess is that the iLO takes care of the server cooling, and now that it is broken, the server overheats.

Is there anyway to fix the iLO without a complete system board replacement?  I tried resetting the security dipswitch on the motherboard (setting switch 1 to ON),  and also removing the battery on the system board for several minutes allowing the BIOS and iLO to reset to defaults -neither of these worked.

Also I cannot even update the firmware of the server BIOS - it tells me it cannot communicate with the iLO sub-system and fails.  So in order to update the BIOS of a server, the iLO needs to be working?

Any suggestions?

NK

[Oscar A. Perez · Accepted solution]

  I've seen iLO becoming extremely busy and ROM reporting POST error 1500 when there is a corrupted AHS file that iLO4 cannot erase.   This issue was fixed in the new iLO4 2.40 but, the problem is how to land 2.40 on that box where iLO4 isn't responding at all.

One way to do this is by flashing iLO4 in direct mode.   Here are the steps:

1) Download the iLO4 2.40 Online ROM Flash Component for Linux ( CP027578.scexe )  :

  https://www.hp.com/swpublishing/MTX-ae600534c9b64bb9a10f0b299c

2) Download the latest SPP ISO image and use the HP USB Key Creator Utility to make a bootable USB flash drive.

3) Copy the CP027578.scexe into the folder  /hp/swpackages  

4) Open the server hood, and turn on the iLO Security Override Switch on the system board (hood has a label showing the exact location of the Dip Switches)

5)  Boot Server into SPP, select Interactive mode and once it gets to the main menu, press ctl+alt+d+b+x to invoke the prompt.

6) Navigate to /hp/swpackages folder

7) Execute:  
       CP027578.scexe --direct     ( <-- notice the two dash characters before the word direct )

8) Once the iLO flash is complete, power Off the server, flip the iLO security override switch back to normal position.

[jlucia_cysd]

Thanks, this helped me a lot. I was having the same issue. I had to fine tune some of the instructions so I will list them below.

6). To navigate to /hp/swpackages for me on SPP 2016.04.0 I used the path /mnt/boot/hp/swpackages

6a). Type su to become super user.

7). Execute with: ./CP027578.scexe --direct

You will likely need to stop the ilo service, but I can't recall the command, it will tell you what to do when you attempt the flash.

Hope this helps, thanks to Oscar Perez for putting these instructuions online, saved me a lot of downtime.

[NJK-Work]

Sorry - I totally forgot I had this post.  I will try your suggestion.

Thanks

NK

[Armand_MX]

I have ilo 2.40 but the issue continue.

Any other solution

[francisco89]

Hello 

Managed to resolve using the latest iLO update :

https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_32179cba6b174fb4a43c47fd45#tab-history

CHeers

[sadegh-molaei]

1. Set the ilo 4 Security Override Switch on the system board to the ON position. The location of the switch is printed on a label located on the inside of the server blade hood cover. On the same maintenance switch, set switch number 3 to the ON position.
2. Download the Smart Update Firmware Maintenance DVD version 9.10 B and later. The DVD ISO image is available for download from the following URL on the Downloads tab:

https://www.hpe.com/

1. Create a bootable USB key containing the contents of the Smart Update Firmware DVD.
2. Download the desired version of the ilo 4 firmware smart component for Linux (cp0xxxxx.scexe where xxxxx is an appropriate 5 digit number).
3. Copy the downloaded ilo 4 firmware to the directory /hp/swpackages on the USB key.
4. Put the server blade back into the enclosure and power the server blade ON. Boot to the USB key containing the Smart Update Firmware DVD and select interactive firmware update.
5. Use the following key sequence to exit out of the Smart Update Firmware Maintenance DVD interface. A command prompt will be displayed:
   
   CTRL + ALT + d + b + x (Keep the CTRL and ALT keys pressed when typing d b x. )
   
   The command prompt takes approximately 30 seconds to be displayed.
6. At the command prompt, navigate to the Smart Update Firmware DVD directory containing the supplemental ilo 4 firmware update by using the following command:
   
   bash-3.1# cd /mnt/cdrom/hp/swpackages
7. Use the following command to unload the HPILO module:
   
   rmmod hpilo
8. Use the following command to execute the ilo 4 firmware update in direct mode:
   
   sh cp0xxxxx.scexe - - direct (This parameter requires two dash ( ) characters.).
9. After the ilo 4 firmware upgrade is completed, power the server blade OFF and set the ilo 4 Security Override Switch on the system board to the OFF position. On the same maintenance switch, set switch number 3 to the OFF position