Query: Need Help on Proper Sequence to replace hot swap hard drive

[HelpYourIT]

Hi, 

Have a Proliant ML350 GEN 10 server with 6 hard drives. Logical Drive 1 is a RAID 1 with two drives, no issues. Logical drive 2 has 4 900GB SAS drives on a RAID 10 configuration. Recently, the following warning in the ADU Report with error lights on them

HPE Smart Array P408i-a SR Gen10 in Embedded Slot : Physical Drive (900 GB SAS HDD) 2I:3:5 Critical:  The physical drive has failed.

HPE Smart Array P408i-a SR Gen10 in Embedded Slot : Physical Drive (900 GB SAS HDD) 2I:3:7 Warning:  Physical drive status: Predictive failure. This physical drive is predicted to fail soon.

HPE Smart Array P408i-a SR Gen10 in Embedded Slot : Physical Drive (900 GB SAS HDD) 2I:3:8 Warning:  Physical drive status: Predictive failure. This physical drive is predicted to fail soon.

HPE Smart Array P408i-a SR Gen10 in Embedded Slot : Logical Drive 2 Warning:  Logical drive status: The current array controller has a bad or missing drive. This logical drive is operating in interim recovery mode with reduced performance and a further physical drive failure may result in data loss depending on the fault tolerance.

So basically, for Logical drive 2 (RAID10 on a 4 drives),: One is dead, the other two have predective failure

Physical drive 5: Failed (Flashing Amber)

Physical drive 6: OK

Physical drive 7: Predictive Failure with the Do Not Remove light on (Flashing Amber)

Physical drive 8: Predictive Failure (Flashing Amber)

I have bought 3 replacement drives and plan to replace them in the following sequences:

Step 1: Remove failed drive 5, and put the new drive in. Wait for the drive to become normal

Step2: Remove the drive 8 (the predictive failure). Wait for the drive to become normal 

Step3: Hopefully the "Do not remove" light will be off by then for drive 7. 

Step4: Remove the drive 7 and put the new drive in 

Can someone help with these steps? Any issues you see or any suggestions for proper steps? My image-based backup not working due to the disk error.

Thanks

[rabindra11sharm]

Dear HelpYourIT

mentioned sequence is right proccess to replace faulty HDD. but there is quastion is bay7 & Bay 8 both HDDs are in same array?? Bay7 and bay8 both HDD belongs from RAID 1+0, confirm first, both hdds are in same RAID 1 or separate?? If in same RAID 1, it might be getting issue as both have PFA, get fiull data back up for recovery as recomended from OEM for any activity on array system. 

Let us know after completion the activity, if you observed any difficulty let us know.

Hope, I could provide you with clear and helpful instructions. If you have any more questions or need further assistance, don't hesitate to ask. I'm here to help!

Have a great day!

[TVVJ]

Hello,

The sequence appers correct. Please have a complete backup of data and then perform any steps. After replacing one drive, please wait until the RAID array has completed re-build and then replace another drive.

Regards,

[HelpYourIT]

Thanks Rabindra.

Yes, drive 5, 6, 7, 8 are in the same array (Array B), configured as logical drive 2, on a RAID 1+0 . 

[rabindra11sharm]

Dear HelpYourIT 

Thanks for your information. 

Please get the full data backup, then performe the activity. firstly replace bay8 HDD and wait for completion the rebuilding proccess and monitor it as bay7 HDD also have PFA, and array is in criticle state So it need special and first attention, after successfully completed replace 2nd disk on bay5 and wait. Once rebuilding completed replace 3rd disk on bay7 HDD. Check array health by smart storage administrator application.

Special suggestion: If possible rebuild bay8 HDD in maintanance mode or pull the server out from production.

for more suggestion or any criticality get back to us.

Hope,  I could provide you with clear and helpful instructions. If you have any more questions or need further assistance, don't hesitate to ask. I'm here to help! Have a great day!

[Tam92]

Hi,

I agree too with the sequence you have mentioned.

Please make sure the rebuild of the drive is complete before you pull out another drive for the replacement.

RAID 10 can withstand 2 drive failures only , provided both the drives are in the different group. 

Data backup is an important step in this case.

Thanks,
TAM

[support_s]

Hello,

Let us know if you were able to resolve the issue.

If you have no further query, and you are satisfied with the answer then kindly mark the topic as Solved so that it is helpful for all community members.

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.