topic Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...) in ProLiant Servers (ML,DL,SL)

Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

kzhang_pamp — Sun, 26 Mar 2023 16:38:43 GMT

Hey,
We have a Prolaint Apollo XL675D Gen10 server with BIOS A47. Recently we ran into this issue with the PCI Bus:

-Unrecoverable I/O Error has occurred. System Firmware will log additional details in a separate IML message entry if possible.

-Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0x84, Device 0x1, Function 0x0). Uncorrectable Error Status: 0x100000

How can we solve this issue? Thanks!

Updates. We tried updating the BIOS from v2.56 (02/10/2022) to v2.68 (02/06/2023) (found here https://support.hpe.com/connect/s/softwaredetails?softwareId=MTX_d43155b2420946b58ca01ffedd).

The problem persists

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

kzhang_pamp — Thu, 23 Mar 2023 16:28:28 GMT

Update the system BIOS from v2.56 (02/10/2022) to v2.68 (02/06/2023) (found here https://support.hpe.com/connect/s/softwaredetails?softwareId=MTX_d43155b2420946b58ca01ffedd).

The problem persists

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

TVVJ — Thu, 23 Mar 2023 15:52:06 GMT

Hello,

You may start by updating the server's BIOS. You may either of the below links based on the operating system:

Online ROM Flash Component:

Windows x64
Linux

Regards,

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

kzhang_pamp — Thu, 23 Mar 2023 17:22:33 GMT

UPDATES

We installed the Linux version system BIOS(Online ROM Flash Component) for our server running Ubuntu 22.04. The situation is the same.

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

BPSingh — Fri, 24 Mar 2023 01:31:26 GMT

Greetings!

It has to be decoded to which device Bus 0x84, Device 0x1, Function 0x0 is pointing to.
This could point to a peripheral card like GPU, Network card, etc..

Please log a support ticket and provide the AHS logs to get this issue investigated.

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

kzhang_pamp — Fri, 24 Mar 2023 02:51:27 GMT

Hey @BPSingh
Thanks for the suggestions, I think you are correct, that location is indeed pointing to the Nvidia A100 GPUS.

I opened a support ticket, however, I cannot even receive the AHS logs as when I hit F10, I get the following "screen of death"!

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

BPSingh — Fri, 24 Mar 2023 03:07:32 GMT

If you have iLO configured on the server, you should be able to download the Active Health System log from there.

https://techlibrary.hpe.com/docs/iss/DL360pGen8/tsg/advanced/Content/151471.htm#:~:text=To%20download%20the%20Active%20Health%20System%20log%20using%20HP%20iLO,for%20the%20last%20seven%20days.

Re: Uncorrectable PCI Express Error Detected. Slot 255 (Segment 0x0, Bus 0xA4, ...)

kzhang_pamp — Fri, 24 Mar 2023 13:15:23 GMT

Hey @BPSingh ,

Thanks for the tips, I was able to successfully upload the AHS in the support ticket.