MSA2324i vDisk Rebuild Problem

[InTech2000]

Hi,

We have an issue with our MSA2324iand need your help. 

Initial Problem

• It looks like we had a couple of disk that had SMART errors, and this made it go into a degraded state.
• The disks were marked as LEFTOVR
• I cleared the metadata on the disks and assigned them as dedicated spares.
• The vdisk started rebuilding - Great.

The rebuild finished however and now the vdisk was showing "Fault" with a "red cross" and the message  "The virtual disk is offline. Disks may be down or missing. Data may be lost."

I tried clearing the metadata again, and added them as spares and disks are showing as :

2 x AVAIL

2 x VDISKSP

The vDisk is a RAID 10 with 10 Disks allocated.

I have all green ticks showing on all the disks in the enclosure - but the rebuild will not start.

I tried reallocatting again, but still it didnt start.

What I am doing wrong? How do I set off the rebuild process manually?

Many Thanks.

[SUBHAJIT KHANBARMAN_1]

You already done the damage and vdisk recovery not possible it seems.

You shouldn't clear metadata in multiple drives when you have QTOF or Offline Vdisks. You should always take help from storage experts before you try anything.

It's possible that in RAID10 vdisk two drives part of same sub-vdisk you have cleared metadata so it lost that mirror set and that's why data recovery not possible.

You can still log a case with HPE support and try your luck.

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

***********************************************************************

[InTech2000]

Hi,

Thanks for replying.

If I add more disks in - would it start rebuilding then do you think?

Also , how do I bring the volume back online manually?

Thanks..

[SUBHAJIT KHANBARMAN_1]

No it will not help.

Try to understand the logic, every drive have one section called metadata which contains vdisk configuration details. If you clear that metadata then that drive lost all details so it will not participate for RAID recovery.

For example, suppose there is one vdisk created with 5 drives in RAID 5 this means 4 data and 1 parity

In this condition if 1 data drive failed then you can get back data because other 3 data drives and 1 parity drive will rebuild failed drive data. But in case back to back two drive failed then we can't recover back data. That's called RAID failure.

In your case also same happened. Only difference is your vdisk was in RAID10 where sub-vdisk RAID set got failed.

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

************************************************************************

[InTech2000]

Hi,

The drive was in Leftover state. The only way to rebuild was to clear the metadata. This did work and it started rebuilding.

After the rebuild, it showed the vdisk as offline.

The config at the moment shows :

Raid 1-1  - Disks 1,2

Raid 1-2  - Disks 1,2

Raid 1-3 - Disks 1,2

Raid 1-4 - Disks 1,2

Spare - 1,2,3

There are no errros on the disks - so I am not sure why its not rebuilding as there are spares available..

Is there a manual way to start the rebuild?

[SUBHAJIT KHANBARMAN_1]

Thanks for sharing the details but you still not able to catch the RAID logic

Your vdisk created with 10 drives and in RAID10 which means you have 5 RAID1 sub-vdisk which are in RAID0 to build the complete RAID10 vdisk.

You see you have 4 sub-vdisk available out of 5 sub-vdisk. Your 5th sub-vdisk is completely missing because those two drives lost their metadata which means one RAID1 sub-vdisk is missing.

That's why complete vdisk shows as offline.

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

************************************************************************

[InTech2000]

Hi,

Apologies I gave you the wrong information.

The original config had 8 disks and had :

Raid 1-1  - Disks 1,2

Raid 1-2  - Disks 1,2

Raid 1-3 - Disks 1,2

Raid 1-4 - Disks 1,2

---------------

After the failure its now showing :

9 Disks (not sure why!)

Raid 1-1  - Disks 1,2

Raid 1-2  - Disks 1,2

Raid 1-3 - Disks 1,2

Raid 1-4 - Disks 1

Spare - Disk 1

I was hoping it would rebuild from the dedicated spare.  There are no quarantined disks.

Is there manual way to do this? 

[SUBHAJIT KHANBARMAN_1]

Can you please share the below output,

show vdisks <Vdisk_name>

Where Vdisk_name will be the vdisk which you are trying to recover

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

************************************************************************

[InTech2000]

#show vdisks vDisk1_R10
Name Size Free Own Pref RAID Disks Spr Chk Status Jobs
Serial Number
---------------------------------------------------------------------------
vDisk1_R10 2398.0GB 0B A A RAID10 8 1 64k OFFL
00c0ff1162050000c9371b4e00000000
---------------------------------------------------------------------------

[SUBHAJIT KHANBARMAN_1]

I would suggest to get a call log with HPE support because without checking MSA log it will be difficult to work on this further.

As vdisk in offline state so we may have to excute below command but that's not safe to run without checking logs otherwise you will face data loss,

trust enable

trust vdisk vDisk1_R10

Until vdisk back to degraded state Vdisk rebuilt wouldn't start.. Any vdisk in QTOF or Offline state wouldn't start rebuild eventhough you have spare configured.

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

*********************************************************************