DL325 Gen10 - NVMe disks on Ubuntu disappear and reinsertion of disks required - 40 servers !

[Stephen Vella]

We have a customer who in 2020 bought a bunch of DL325 Servers for a Ubuntu containrised platform. They are running Ubuntu 18.04 LTS. configuration of each server node is as per below. Customer started having issues inthe past 3 months. The 2 NVMes are configured in Software RAID1. randomly disks are failing, breaking the RAID and sometimes both fail, hence server crashes. Mind you nothing is reported in iLO. The solution is to follow the procedure to unplug and reinsert the disk and it comes back up (no reboot needed in cases when one disk fails). We are not getting proper support on this, and we cannot open a case on each server S/N as L1 ask for, which is ridiculous for a server farm. customer is now angry and started even asking to replace all NVMes !!

 Product  Description Qty

P04662-B21 HPE DL325 Gen10 NVMe SFF CTO Server 1

P16922-L21 HPE DL325 Gen10 AMD EPYC 7262 FIO Kit 1

P19043-B21 HPE 32GB 2Rx4 PC4-2933Y-R Smart Kit 2

P10222-B21 HPE 1.6TB NVMe x4 MU SFF SCN DS SSD 2 665240-B21 HPE Ethernet 1Gb 4-port 366FLR Adapter 1

865428-B21 HPE 800W FS Univ Ht Plg LH Pwr Sply Kit 2

J6X00A HPE SDGRD-SDGRD 277AC_380DC 2.0m Jpr Crd 2

BD505A HPE iLO Adv 1-svr Lic 3yr Support 1

P04983-B21 HPE DL325 Gen10 RPS Enablement Kit 1

734807-B21 HPE 1U SFF Easy Install Rail Kit 1

H7J34A3 HPE 3Y Foundation Care 24x7 Service 1 H7J34A3-RA6 HPE ProLiant DL325 Gen10 Support 1

[SanjeevGoyal]

Hello,

It will be very difficult isolate the issue without the proper logs and issue in 40 servers.
Hence, I would suggest, you have to log a proper case with HPE, and share the appropriate logs for further analysis.

If you feel this was helpful please click the KUDOS! thumb below!   
Regards,

[SanjeevGoyal]

Hello,

Please ignore the pervious post.

Follow below one.

Please share the NVMe hard disk model number and current firmware.

I hope server is running with the latest bios and SA controller & hard disk firmware.

If you feel this was helpful please click the KUDOS! thumb below!   
Regards,