POST Error: 338-HPE RESTful API Error - Unable to communicate with iLO FW. BIOS configuration

[Server-Support]

Hi All,

I got this email alert after I upgrade the firmware using HPE SPP 2019.03.1(4 May 2019) and then followed by iLO firmware 2.70 May 07 2019

POST Error: 338-HPE RESTful API Error - Unable to communicate with iLO FW. BIOS configuration resources may not be up-to-date. Action: Restore Manufacturing Defaults in RBSU.

Upon further digging in the iLO screen and diagnostic info, I can see there is some error message:

Error in iLO?

How can the error above be rectified?

Any help would be greatly appreciated.

[MV3]

Hi,

Looks like the Nand format is required to address this issue.

I would suggest you to perform the below shared link and check if it fixes the issue.

Advisory Link : https://support.hpe.com/hpsc/doc/public/display?docId=a00048622en_us

Follow the Steps to perform a NAND format via the iLO GUI.

cheers

[Server-Support]

@MV3 Is there anything that will be lost or reset to factory default after executing the reset steps?

Do I need to backup something first?

[MV3 · Accepted solution]

HI Server-Support,

You need not to back up anything if you are tring to only reset the nand chip from GUI.

Cheers

[Server-Support]

Thank you @MV3 , I assume there will be no data loss in the system drive or in the Server BIOS?

Can I do it remotely via iLO?

[PDP-Fan]

Re-formatting the NAND flash chip from the ILO webinterface may help for a moment. I had flash chip errors on many G8, G9 and G10 servers and every time, the error came back after a while.

The problem is the NAND flash chip itself. Instead of using a pluggable SD card, HP solders the chip to the mainboard. That means, you have to replace the mainboard, except you find someone who is able to solder a new micro BGA flash chip to your board.

A definitive check if the flash chip is bad or not, is to update the Intelligent Provisioning. This software is the reason why the chip exists at all, it replaces the old CD/DVD smart start. If the update works, your chip is good for another couple of years (hopefully). If it fails....  read the middle section again

Why does the chip fail? Nand flash chips have a limited lifetime. Especially writing to them wears them out. In the beginning, the Gen8 servers died like flies because iLO was writing logfiles way too often to theose chips. After a iLO firmware update, it got better. It seems though, the problem still exists, because even Gen10 servers can dvelop that problem.
Good advice to HP: Please use a removable SD card or put the chip on a small replaceable PCB, so that not the entire mainboard must be replaced. I know you can do that, because you did exactly that on the BL460 Gen9. But only there and never again.

[TomJ802]

This seems like somewhat of a known issue - I stumbled over references to it while researching the purchase of a G11 server.

hpe nand chip failure - Google Search

Hopefully it's been resolved in the current gen...

Edit - I agree that this soldered-in design is flawed.  HP used to be known for above average build quality and reliability...  Now everything seems to be manufactured with a focus on cost savings these days...

[PDP-Fan]

I don't know if they solved the problem on G11 servers, but I know that we already had G10 servers with a faulty flash chip. It takes about 5-6 years until it falis, just perfect for a 3-5 year warranty.

I don't understand HP... all they need to do is to use a pluggable MicroSD card and a socket for it on the board. The chip they used is already a SD card but one to solder. They basically just have to change the PCB footprint from BGA-chip to MicroSD socket.