MSA Storage
1748003 Members
4508 Online
108757 Solutions
New Discussion

Re: Replacing two corrupted drives in a RAID 50 on MSA 2012sa

 
Xavier85
Occasional Advisor

Replacing two corrupted drives in a RAID 50 on MSA 2012sa

Hello all,

I need help determining the course of action to restore my MSA functionality (12 drives in a RAID 50 with 1 vdisk on a MSA 2012sa).

 

Slot 4 had an amber light and showed as missing in the vdisk overview, so I changed it.

However, the MSA experienced a hard drive failure (slot 2) during the reconstruction of the new drive (slot 4). 

 

As a result the two drives (the one that failed and the one reconstructing) had amber lights, and the vdisk was "offline".

I cleared the metadata on those two drives (their status went from "L" to "A") to get the vdisk to "quarantine". Afterwards, I couldn't dequarantine it (returns "request failed "1" " when I try to dequrantine.)

 

I removed both slots 2&4 to install the old drive from slot 4 (which was malfunctioning) => the vdisk dequarantined and went to "critical". The vdisk is back online but the server can't access some of the folders and tells me the data is corrupted.

 

I dont understand why reinstalling the old drive helped the vdisk go back to "critical" and why the LED lights for that slot is now green !  

It makes me worry that maybe the issue doesnt come from the drive but from somewhere else. The other LED indicators and SMU indicators are green for controller and enclosure status.

 

I ordered two new drives to replace slot 2&4.

- What is the procedure for changing two drives at once? (while minimizing data loss)

- Should I be looking at non-hard drive related issues?

- Can I "format" a drive before adding it to vdisk spares? (I'd like to do that so I can re-use the new drive that was affected during this incident since the drive should be fine from a hardware point of view.)

 

Thank you very much, and please let me know if I need to complete my description.

Xavier

3 REPLIES 3
Dennis Handly
Acclaimed Contributor

Re: Replacing two corrupted drives in a RAID 50 on MSA 2012sa

>What is the procedure for changing two drives at once? (while minimizing data loss)

 

If you lose two drives in RAID 5, it's time to get out your backups.

Xavier85
Occasional Advisor

Re: Replacing two corrupted drives in a RAID 50 on MSA 2012sa

Okay, I will replace them for sure since I dont have a choice. I couldn't find whether there is a special procedure, maybe there isn't tho.

Xavier85
Occasional Advisor

Re: Replacing two corrupted drives in a RAID 50 on MSA 2012sa

Just to bring some closure to this. The drive had not totally failed which is why the vdisk was up when I had the drive plugged in. 

I tried a new reconstruction, which failed again before finishing because the same drive failed again.

 

After removing the failling drive, I was left with 10 good drives, one functioning drive but only partially reconstructed, I cleared the cache on the partially reconstructed drive and I was able to recover the critical mode.

 

While waiting for the new drives to arrive, I ran chkdisk and checked the directory and found that a lot of the data was not lost (probably thanks to the fact this is a RAID 50).

 

Then it was just a matter of replacing the second drive which I did and the system reconstructed just fine.