Upgrade to SPS v04.01.04.381 on Blade 460c Gen 10 breaks server

[Stephan G]

Hi everyone,

due to security reasons we upgraded the Intel SPS to the lastest version.
After the next restart - the server does not even boot to the bios. The screen stays black and the the firmware of the innovation engine throws an exception. 

If we try to update the firmware (IE) from ILO we get the error:

The last attempt to update or upload firmware was not successful. Make sure you are using a valid, signed flash file and try again. If you are attempting to install a component, first upload it to the iLO Repository then add it to the Installation Queue.

No matter which bin we are uploading - different ppl different clients.

We cannot use any recovery media as the server does not even come that far. We are stuck in "pre POST"

Anyone that can give us a hint how to proceed (we will open up a case shortly).

Best regards

Stephan

[Torsten.]

How exactly did you do the update?

Try to use this

https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_8d1148a85dde488ebeec6d6aa4#tab-history

and use Firmware Update from ILO.

[Stephan G]

Hi,

we deployed it by installing the Win Version then restarted the server.

We also tried to update it from ILO. There we get the error message i have posted.
It does not matter if we deploy the IE or the SPS. But we used the extracted BIN Files as there are several posts around that recommend that. Is the fwpkg also possible?

I will try that later. At the moment it does not matter as the server is dead.

Best regards

Stephan

[Stephan G]

HPE will replace the board of the affected blade.

But the story continues. We have 2 other DL360 Gen10 servers. Also tried to patch with SPS 4.1. Same day like the blade.
It worked with 4 DL360 g10 before that!
The prereqs are fulfilled.

We install the SPS from Windows - after that the ILO states:
The overall security state of the system is at "Risk".

The restart of the servers went well - Firmware cannot be installed with some error

The update task "Update-545D46 Server Platform Services (SPS) Firmware 04.01.04.3" has changed state to exception.

Initial Update 06/29/2020 17:34:25 Event Class 0x23 Event Code 0x1DF

iLO detected invalid Server Platform Services firmware.

Initial Update 06/29/2020 18:22:31 Event Class 0x20 Event Code 0x3

Since then we have the warning:
The iLO health monitoring status of the device / adapter located in Embedded is not responsive

The difference between the servers that worked and the ones who are not - the one that worked had a higher patching level regarding BIOS, IE and so on - but should make no difference if the prereqs are fulfilled?!?

[Stephan G]

So bottomline:

DO NOT INSTALL THE SPS Without installing the newest Service Pack for Proliant on the server first.

3 dead servers - still ongoing.

I do not need an Innovation Engine in my servers - running engine would be better.
Keep the crap out of the servers and focus on the essential stuff. TY

[kdepew]

There is an issue with SPS v04.01.04.381 for HPE Gen10 servers when updating to this revision of firmware from SPS v04.00.00.288 or earlier.  When flashing to SPS v04.01.04.381 on a system that contains SPS v04.00.00.288 or earlier, the system will be left in an inoperable state and cannot be recovered.  The issue does NOT occur if the system was using an SPS firmware revision later than v04.00.00.288 prior to the flash update.  Due to this issue, the flash component for SPS v04.01.04.381 has been removed from the HPE Support site for download.  A new version is currently under validation and will be made available in the near term.   And Advisory is also being released.  Note that the new SPS firmware update will still be v04.01.04.381 because this versioning is from Intel.  This will be a new revision of the flash component.  The original issue was with how HPE built the latest Intel SPS FW release.

SPS v04.01.04.381 was Intel's latest SPS FW and included security updates which Intel publicly disclosed on June 10, 2020.  The issue was with the build of the FW revision by HPE.  This is being corrected.  This release, nor this issue, was related to the Innovation Engine or its firmware.