PCI Express Error (Embedded device, Bus 0, Device 1, Function 0, Error status 0x00000054)

[namuha]

Proliant XL230a gen9  is crashing frequently with error:

Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 1, Function 0, Error status 0x00000054)"
Unrecoverable System Error (NMI) has occurred. System Firmware will log additional details in a separate IML entry if possible 

"PCI Bus Error (Slot 0, Bus 0, Device 1, Function 0)"

ILO Version 2.75

System BIOS - U13 v2.76

which device is Slot 0, Bus 0, Device 1, Function 0

thanks for help

[ramawatar]

Hi ,

This error is pointing to device installed in Slot 0. 

Please check what the device installed in Slot 0, this seems to be a hardware issue. 

Login in ILO GUI and go to Device inventory to see the devices installed in Slot 0, you might need to log a support case to investige the issue further. 

[namuha]

Thank you for your support ramawatar,

The only Embedded Device that is shown in the Device Inventory :

• Embedded HPE Ethernet 1Gb 4-port 331 FLR Adapter - NIC Firmware Version 20.14.62
• PCI-E Slot 1 empty
• PCI-E Slot4 empty
• Unknown HPE Ethernet 1Gb 4-port 331 FLR Adapter, 629135-B22, 789897-001, %CB629XXXX, 00, 20.14.62, OK

[ramawatar]

hi,

Thanks for the update that there is nothing installed in the Slot 0.

As i said initially this seems to be hardware fault issue and you need to log a support case.

[namuha]

Suddenly there is no active warranty on this System any more.

Also we had the same issue on another identic machine in the same chassis.

Nobody in our company had those errors before and we have not only two of those Servers.

Thank you for your support

[ramawatar]

is it possible for you to share me AHS from this server to debug more.

if this can be resolve with the driver firmware then i can help you

[namuha]

i could share AHS logs.

Firmware on the System are on stand

I found no problems as i was onsite.

[ramawatar]

Hi,

Do you need any  further update or should we close this thread as you have mentioned that there was no issue found when onsite. 

if the firmware and drivers are updated and still you see the PCI express error then there is hardware issue which can be rectifiy/resolve after the support case log.

[namuha]

linux generates following:

[380990.127185] Hardware name: HP ProLiant XL230a Gen9/ProLiant XL230a Gen9, BIOS U13 10/21/2019
[380990.127218] Call Trace:
[380990.127239]  [<ffffffffb957b416>] dump_stack+0x19/0x1b
[380990.127263]  [<ffffffffb9574a0b>] panic+0xe8/0x21f
[380990.127311]  [<ffffffffb8e9b80f>] nmi_panic+0x3f/0x40
[380990.127335]  [<ffffffffc05e63df>] hpwdt_pretimeout+0x6f/0xb0 [hpwdt]
[380990.127367]  [<ffffffffb958593c>] nmi_handle.isra.0+0x8c/0x150
[380990.127393]  [<ffffffffb9585bc4>] do_nmi+0x1c4/0x460
[380990.127418]  [<ffffffffb8e30e74>] ? iommu_shutdown_noop+0x4/0x10
[380990.127442]  [<ffffffffb9584cc9>] nmi+0x109/0x158

[ramawatar]

Hi,

As you have already confrmed that all the drivers/frimware are up to date but still you are getting PCI express error then there is something which needs to look into the hardware. 

you need to log a support case for this issue.