iLO Upgrade | Embedded Flash/SD-CARD: Embedded media initialization failed due to media write-verify

[Anthony_Pap]

After upgrading the iLO on one of our Servers I am getting the following message:

“Controller firmware revision 2.10.00 Embedded media initialization failed due to media write-verify test failure”

and the indication Degraded on page Overview of iLO Health Status.

I 've found the link below from HPE documentation:

https://support.hpe.com/hpesc/public/docDisplay?docId=a00048622en_us

After checking on the first method suggested, I noticed a particular warning that has to do with BIOS:

“WARNING: Formatting the Embedded Flash erases all data on the iLO partition and cannot be undone. External providers (like BIOS, Intelligent Provisioning, OneView and FLM) will need to be re-configured.”

Is it safe to proceed in such an action, given that there is a certain configuration on the RAID controller and maybe other additional BIOS settings, that could be affected. This particular server is really important to risk any further issues. Could you please suggest any of the other indicated methods as "safer", in terms of not risking to lose crucial BIOS configuration?

The server is a HP ProLiant ML310e Gen8 v2 and the firmware was updated from version 1.40 to version 2.78.

Thanks in advance,

Anthony Papathanasiou

[Torsten.]

Formatting the NAND does not affect the RAID.

Follow this advisory

https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00048622en_us

[Ihaqueit]

Thenupgrade ILO through the web interface

[Anthony_Pap]

Thank you very much for your help and concern.

My question is based on the warning message below, which is part of NAND Format Method 1: Use the iLO 4 GUI :

“WARNING: Formatting the Embedded Flash erases all data on the iLO partition and cannot be undone. External providers (like BIOS, Intelligent Provisioning, OneView and FLM) will need to be re-configured.”

How risky could this procedure be in terms of motherboard BIOS settings or, even, functionality?

Please keep in mind that the server is a DC and the owner of FSMO roles in the AD.

Thanks and regards

[Ihaqueit]

Can you generate an AHS report? If not, then it's probably not a NAND issue and formatting won't help. Are there any error messages during POST? There are several different issues that can cause a degraded iLO that aren't corrected by formatting the NAND.

You can perform the NAND format with the server off, iLO runs independent of the server, it just needs power.

Correct it just erases the NAND flash for iLO and formats it. Good luck on that actually fixing your problem. Those NAND chips fail and there’s nothing a format can do.

[Ihaqueit]

Actually, this is the point, but even if there is other data on the SD card, they will not be erased.
Only iLO partitions are erased.

Therefore, it is possible to perform the above operations even when the server is running.

However, if you are booting ESXi from an SD card,it is strongly recommended to shut down the OS and then format it, so let's do so.

Above, when iLO is Degraded, it was a method of formatting only the management area of iLO of SD card.

[ManBha]

Hello,

In most cases, this should not cause any production impact.

You can also proceed with the system board replacement in case of any issues.

Thanks.

[Anthony_Pap]

Thank you all very much for your answers.

Unfortunately I couldn't find any definite or straightforward.

The way I see it is that the procedure is rather risky, and according to the last answer, there is a chance that the system board should be replaced.

Since it's a rather old Server, I guess it's better to leave the situation as it is, until I come up with a "safe" way to overcome the problem reported.

Thanks again and regards.

[ScottRCI]

Hi @Anthony_Pap  did you ever get this resolved?  We have one of these as well, and I did a board replacement (although with a used board) and the issue persists.

Did you ever find out if this actually causes issues ???  We do NOT use the SD card, so just wondering if we are ok.

[scamposgarci]

I have the same ILO memory issue, but it also seems to make not able to log into my system BIOS.

I tried to format the NAND with all the methods listed in https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00048622en_us and the output seems to be possitive according to the messages displayed, but it never shows the message in the ILO log section "Embedded Flash/SD-CARD: One or more storage devices have been formatted". 

I tried:

• via the iLO GUI
• Windows (Remote Format)
• Linux or VMware (Local Format)

In addition how can I contact wioth HP support so I would get a replacement board? It seems that you have to have a support contract even for getting a quote, what is crazy considering I'm a hobbyist end user and my server is for personal use.

[ScottRCI]

You can also try and flip the hardware switch for iLO security (DIP Switch 1) - in some cases (where there is not an actual board issue) that can help.  It did not help me.  I tried another board from eBay but alas, it had the same issue and I had to take it all apart again to return.  I ended up selling the entire server and moving on.

If it is just for hobby, I would just ignore it.  Does not seem to have any actual effect on the server itself (as long as you are not running VMware and require that nand/flash).

[Torsten.]

>> (as long as you are not running VMware and require that nand/flash)

VMware does not require this bult-in flash, but unfortunately if the built-in flash has failed the system cannot access the SD-card slot too.

[scamposgarci]

@ScottRCI @Torsten. 

Thanks for the tip of the dip switch, but just tried and it didn't worked.

You are right, so far my server is still usable but I'm stuck with the BIOS Setup I did ages ago. IE the booting device is the onboard microSD port what is not ideal but I have managed to get this to work with my system installed in the SSD.

With the error already present I upgraded the setup last year. (I bought it seccond hand and were pretty cheap)

• 16gb of RAM
• 2.5" SSD HDD in optical bay
• Intel(R) Xeon(R) CPU E31260L @ 2.40GHz

Anyway it would be great to get rid of the BIOS problem. And what annoys me most is that in the envent logs, the attempt of format the embeded flash is not shown. I still feel like I'm missing something.

[sgunelius]

I've seen similar issues on quite a few of our DL360p Gen8 servers and none of the formatting methods addressed the problem.  The only fix was to replace the motherboard because the iLO & NAND are integrated.  Fortunately we had these under extended contract and needed fully functional iLO to use with OneView, iLO Amplifier and InfoSight.

[scamposgarci]

one of my complains is that HP does not provide me not even a quote for this motherboard replacement since I have no support contract in place. 

But as a hopme user it doesnot have much sense

[Ihaqueit]

Contact with reseller.

[scamposgarci]

Sorry, what reseller you mean?

[Ihaqueit]

Genuine product must be buy via reseller, the reseller contact with
distributor or HPE and provide you the replacement.