System Power Fault Detected MicroServer Gen8

[Sukuoru]

Integrated Management Log:

System Power Fault Detected (XR: 10 00 MID: FF F5 FE 01 FF FF FF 06 06 00 00 02 00 05 80 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00)

iLO Event Log:

04/12/2022 04:30 04/12/2022 04:30 1 Server power removed.
04/12/2022 04:29 04/12/2022 04:29 1 Embedded Flash/SD-CARD: Restarted.

Note: I am not using a MicroSD card, unsure of that errors origin. EDIT: Appears the Embedded flash message shows even for past graceful restarts, so probably a red herring.

Active Health Log 

Firmware Info:

iLO 2.79 Nov 16 2021 System Board

iLO 2.80 Jan 25 2022 System Board

Intelligent Platform Abstraction Data 0.00 System Board

Intelligent Provisioning 1.62.14 System Board

Redundant System ROM J06 11/02/2015 System Board

Server Platform Services (SPS) Firmware 2.2.0.31.2 System Board

System Programmable Logic Device Version 0x06 System Board

System ROM J06 04/04/2019 System Board

System ROM Bootblock 02/04/2012 System Board

I would appreciate any direction on where to go with this, if it was perhaps a fluke. I got this server secondhand about a month ago, upgraded hard drives 2 weeks ago, and upgraded the CPU 1 week ago.

EDIT: There was an iLO update available, so I did perform that upgrade. Can't seem to find any other available updates.

[support_s]

System recommended content:

1. Advisory: HOT FIX: HPE ProLiant DL580 Gen9 - Broadwell based DL580 Gen9 Systems may generate System Power Fault Detected messages

2. HP ProLiant MicroServer Gen8 User Guide

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Sukuoru]

That advisory is for the wrong Generation, I am using a Gen8 with an Intel(R) Xeon(R) CPU E3-1265L V2. But it did lead me to find a newer iLO version.

Manual only mentions that the red lights flashing (I did see after being informed the server was down) is for a power fault, and it says to check the logs... which all just say power fault. Nothing on the system is in a degraded state currently.

[Sukuoru]

After 14 days of uptime, it happened again. Server went down, light at the bottom of the unit flashing red. Log shows the same error number, now with a count of 2. Around the time it went down, there was no load on the server, it was just idling.

Integrated Management Log

System Power Fault Detected (XR: 10 00 MID: FF F5 FE 01 FF FF FF 06 06 00 00 02 00 05 80 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00)

ILO Event Log:

1021
04/25/2022 20:18 04/25/2022 20:18 1 Server power restored.
1020
04/25/2022 20:18 04/25/2022 20:18 1 Server reset.
1019
04/25/2022 19:23 04/25/2022 19:23 1 Server power removed.

I have nothing to go on, server worked well then a lot of changes at once. I can only assume a fault in the CPU (as it was replaced with a second hand unit) or possibly the Power Supply (4 spinning disks, 1 ssd, 1 usb, higher powered CPU) but I have no idea. Any suggestions?

I considered swapping in the OEM CPU, and waiting to see if this error occurs again but I am not sure that would give much direction. If the fault went away, it could still be the PSU. Everything in ILO states "Good".

[Sukuoru]

Happened again, 6 days of uptime. Same error. In contact with the seller of the CPU, per their advice I have the OEM CPU in now to see if the error will resurface. If there was a way to translate that error code and get any information that would be great.

If it happens again after this, I will order a PSU that has been recommended for this particular server. I have nothing to go on, any advice beyond my current plan would be wonderful.

[TVVJ]

Hello,

As the issue persists even after the BIOS being at the latest version, please contact HPE Support and log a case for further assistamce.

Regards,

[h0neybadger]

OP, I'm having this same error and byte sequence. What was you resolution to this issue?

[Sukuoru]

Prior to getting this error I had upgraded the CPU. After HPE support recommended replacing the motherboard which is unrealistic for such an old device, I continued attempting to troubleshoot.

Most recent attempt was cleaning the pads under the replacement CPU with Isopropyl Alcohol and cotton swabs. I haven't had the error in a month. Next attempt will most likely be an aftermarket PSU. Really just throwing things at the wall hoping they stick. Have heard of others using a Seasonic SS-250SU.

[h0neybadger]

Thanks for the info. I recently got the error message with the oriinal CPU (the G2020T) and everything stock (as far as I'm aware because I bought second hand). I'm also planning a CPU upgrade the the Xeon, so should still be in spec for the power supply.

What's annoying is that as far as I can tell HPE does not have public documentation on decoding the MID byte sequence. I've seen in other threads here with different byte sequences where HPE staff have provided extra information decoded from the byte sequence (eg. insufficient voltage). So I expect that the byte sequence were getting has more specifics on where the failure is coming from.