Uncorrectable PCI Express Error after power loss - HP Proliant DL380e

[Shalak]

I experienved power loss, and after booting, iLO shows Critical alert on BIOS/Hardware and the following logs:

"ID","Severity","Class","Last Update","Initial Update","Count","Description",
"994","Critical","PCI Bus","07/23/2024 09:18","07/23/2024 09:18","1","Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 0, Function 0, Error status 0x00100000)",
"993","Critical","System Error","07/23/2024 09:18","07/23/2024 09:18","1","Unrecoverable System Error (NMI) has occurred.  System Firmware will log additional details in a separate IML entry if possible",
"992","Caution","POST Message","07/23/2024 09:17","07/23/2024 09:17","1","POST Error: 1792-Slot X Drive Array - Valid Data Found in Cache Module. Data will automatically be written to drive array.",
"991","Repaired","Network","07/19/2024 14:22","07/19/2024 14:21","7","Network Adapter Link Down (Slot 0, Port 2)",
"990","Critical","Network","07/19/2024 14:22","07/19/2024 14:21","7","Network Adapter Link Down (Slot 0, Port 2)",
"989","Repaired","Network","07/09/2024 14:35","07/09/2024 14:35","6","Network Adapter Link Down (Slot 0, Port 1)",
"988","Repaired","Network","07/09/2024 14:35","07/09/2024 14:35","7","Network Adapter Link Down (Slot 0, Port 2)",
"987","Repaired","Power","07/09/2024 14:34","07/09/2024 14:29","1","System Power Supplies Not Redundant",
"986","Repaired","Power","07/09/2024 14:34","07/09/2024 14:29","1","System Power Supply: Input Power Loss or Unplugged Power Cord, Verify Power Supply Input (Power Supply 2)",
"985","Critical","Network","07/09/2024 14:35","07/09/2024 14:29","7","Network Adapter Link Down (Slot 0, Port 2)",
"984","Critical","Network","07/09/2024 14:29","07/09/2024 14:29","1","Network Adapter Link Down (Slot 0, Port 1)",
"983","Repaired","Network","07/02/2024 19:10","07/02/2024 19:10","1","Network Adapter Link Down (Slot 0, Port 2)",
"982","Repaired","Network","07/02/2024 19:10","07/02/2024 19:10","1","Network Adapter Link Down (Slot 0, Port 1)",
"981","Repaired","Network","07/02/2024 12:18","07/02/2024 12:18","1","Network Adapter Link Down (Slot 0, Port 1)",
"980","Repaired","Power","07/02/2024 12:17","07/02/2024 12:17","1","System Power Supplies Not Redundant",
"979","Repaired","Power","07/02/2024 12:17","07/02/2024 12:17","1","System Power Supply: Input Power Loss or Unplugged Power Cord, Verify Power Supply Input (Power Supply 2)",

Device 0:0:0 appears to be the bridge:

# lspci -vvv -s 00:00.0
00:00.0 Host bridge: Intel Corporation Xeon E5/Core i7 DMI2 (rev 07)
	Subsystem: Hewlett-Packard Company Xeon E5/Core i7 DMI2
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 0
	NUMA node: 0
	IOMMU group: 15
	Capabilities: [90] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag- RBE+
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L1, Exit Latency L1 <16us
			ClockPM- Surprise+ LLActRep+ BwNot+ ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown, Width x0
			TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		RootCap: CRSVisible-
		RootCtl: ErrCorrectable- ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BCD, TimeoutDis+ NROPrPrP- LTR-
			 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS- LN System CLS Not Supported, TPHComp+ ExtTPHComp- ARIFwd-
			 AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, ARIFwd-
			 AtomicOpsCtl: ReqEn- EgressBlck-
		LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
			 EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
			 Retimer- 2Retimers- CrosslinkRes: unsupported
	Capabilities: [e0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [144 v1] Vendor Specific Information: ID=0004 Rev=1 Len=03c <?>
	Capabilities: [1d0 v1] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [280 v1] Vendor Specific Information: ID=0004 Rev=2 Len=018 <?>

My backup is currently broken, waiting for hardware replacements, so I must be extremely careful of my next steps.

What should I do? I think that simple reboot may clear the issue, but at this stage I'm afraid of not being able to POST.

[support_s]

System recommended content:

1. Advisory: HP ProLiant DL380e Gen8 Server - Excessive Throttling of Processors and Memory May Occur with HP Integrated Lights-Out 4 (iLO 4) Firmware Version 2.00 Through Version 2.22

2. HP ProLiant DL380e Gen8 Server Maintenance and Service Guide

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[MV3 · Accepted solution]

Hi,

Based on the above output, I don't observe any real failure. It seems that rebooting the host should resolve the issue.

If the problem continues, please open a support case with HPE for analysis of AHS logs.

Cheers...