topic Re: Server hangs after RAID-6 single disk failure in Servers - General

Server hangs after RAID-6 single disk failure

Martin42 — Mon, 06 May 2024 04:06:14 GMT

We have HPE servers with P420i v8.32, P840 v7.00 and P440ar v7.00 and Ubuntu Linux. Each of these have RAID 6 arrays with 6-8 SSDs. On three occasions, the following has happened: A single disk in this RAID has failed, and after a little while, like 30 mins, the servers hangs. When we restart the server, the server starts up in degraded mode and works well.

What is the reason for the server hanging? It becomes inaccessible over SSH, and all systems on it stops. Is this a feature of the RAID-controller, or is it the file system in Ubuntu Linux that stops the disk? Or is it the disk driver? Have anyone else had this problem?

Re: Server hangs after RAID-6 single disk failure

shiva_jr — Mon, 16 Sep 2024 09:30:19 GMT

Hi @Martin42,
At first, the hang may be due to something else. RAID 6 has minimum redundancy of two drives. For one disk failure, the RAID won't trouble, it tries to rebuild.

Did you replace the failed disk?
Is the RAID working fine now?
I believe, now you experience no hang status. If not-
a. Check the RAID controller logs
b. Check dmesg and syslog
c. If SSH console of the OS is not working. Please check the OS is booting or not, the network and ssh service so that you can get the root cause from OS perspective.

Regards,
Shiva_JR

Re: Server hangs after RAID-6 single disk failure

Martin42 — Mon, 16 Sep 2024 09:27:42 GMT

@shiva_jr Thank you for the answear! How can I check the RAID controller logs? It works fine after a reboot, but does require a reboot to start again after a disk failure.

Re: Server hangs after RAID-6 single disk failure

Mr_Techie — Mon, 16 Sep 2024 09:26:50 GMT

@Martin42

You can use the HPE Smart Storage Administrator (SSA) or the HPE command-line utilities hpssacli or ssacli. These tools allow you to access detailed information about the RAID controller and the state of the arrays.

Hope this helps!