MSA 2040: The disk group is quarantined.

[repa1]

Hi,

we have an MSA 2040 with a disk group quarantined and one degraded. when we check the disks, all 24 disks are online and "OK".

We already restarted the controllers, but we're not getting this degraded / quarantined status away, even if all disks are online (and we did not replace them)

Anyone has an idea?

# show disks
Location Serial Number Vendor Rev Description Usage Jobs Speed (kr/min) Size Sec Fmt Disk Group Pool Tier Health
---------------------------------------------------------------------------------------------------------------------------------------------------
1.1 0DGHLSHF HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.2 05V2SKVZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.3 05V2SPTZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.4 05V2VU3Z HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.5 05V2WTJZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.6 05V33T7Z HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.7 0DGLYEKH HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.8 05V30NVZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.9 W8GDJTJX HP HPDC SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.10 05V2SUMZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.11 05V30N4Z HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.12 05V30LLZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.13 05G1V3TZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.14 05G1WK7Z HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.15 05G22YRZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.16 05G230BZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg02 dg02 N/A OK
1.17 WBN03YG3 HP HPD1 SAS LINEAR POOL 10 1800.3GB 512e dg04 dg04 N/A OK
1.18 WBN02BDH HP HPD1 SAS LINEAR POOL 10 1800.3GB 512e dg04 dg04 N/A OK
1.19 WBN03CRV HP HPD1 SAS LINEAR POOL 10 1800.3GB 512e dg04 dg04 N/A OK
1.20 WBN0338N HP HPD1 SAS LINEAR POOL 10 1800.3GB 512e dg04 dg04 N/A OK
1.21 W4615LM3 HP HPD8 SAS MDL LINEAR POOL 7 2000.3GB 512e dg03 dg03 N/A OK
1.22 W4615LKC HP HPD8 SAS MDL LINEAR POOL 7 2000.3GB 512e dg03 dg03 N/A OK
1.23 W4616HDV HP HPD8 SAS MDL LINEAR POOL 7 2000.3GB 512e dg03 dg03 N/A OK
1.24 W4616GXT HP HPD8 SAS MDL LINEAR POOL 7 2000.3GB 512e dg03 dg03 N/A OK
---------------------------------------------------------------------------------------------------------------------------------------------------
Info: * Rates may vary. This is normal behavior. (2019-05-21 17:34:42)

Success: Command completed successfully. (2019-05-21 17:34:42)

# show disk-groups
Name Size Free Class Pool Tier % of Pool Own Pref RAID Disks Spr Chk Status Jobs Job% Serial Number Spin Down SD Delay Sec Fmt
Health Reason
Action
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
dg01 5395.6GB 0B Linear dg01 N/A 100 A A RAID6 8 0 512k FTOL 00c0ff26810700007769605600000000 Disabled 0 512n
OK

dg01 1798.5GB 37.7MB Linear dg01 N/A 100 A A RAID10 4 0 512k QTOF 00c0ff25c6e90000b86d8b5a00000000 Disabled 0 512n
Fault The disk group is quarantined.
- Look for events in the event log related to quarantine (172, 485, or 590) and follow the recommended actions for those events.
dg02 5395.6GB 0B Linear dg02 N/A 100 B B RAID6 8 0 512k FTDN 00c0ff2680c00000b769605600000000 Disabled 0 512n
Degraded One disk in the RAID-6 disk group failed. Reconstruction cannot start because there is no spare disk available of the proper type and size.
- Replace the disk with one of the same type (SAS SSD, enterprise SAS, or midline SAS) and the same or greater capacity. For continued optimum I/O performance, the replacement disk should have performance that is the same as or better than the one it is replacing.
- Configure the new disk as a spare so the system can start reconstructing the vdisk.
- To prevent this problem in the future, configure one or more additional disks as spare disks.
dg03 3996.7GB 62.9MB Linear dg03 N/A 100 A A RAID6 4 0 512k FTOL 00c0ff268107000018dde15a00000000 Disabled 0 512e
OK

dg04 3597.0GB 88.0MB Linear dg04 N/A 100 B B RAID10 4 0 512k FTOL 00c0ff2680c00000b6a2755a00000000 Disabled 0 512e
OK

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully. (2019-05-21 17:35:16)

[Shawn_K]

Hello,

You are going to need to open a support case to help resolve your issue. We cannot fully troubleshoot your problem in a forum.

From what I have seen from the breif log snippet your drives are not correct.. You show the following drives for dg01

1.1 0DGHLSHF HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.2 05V2SKVZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.3 05V2SPTZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.4 05V2VU3Z HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.5 05V2WTJZ HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK
1.6 05V33T7Z HP HPD5 SAS LINEAR POOL 10 900.1GB 512n dg01 dg01 N/A OK

Your disk-group shows that it should have 4 drives instead of the 6 you are showing.

dg01 1798.5GB 37.7MB Linear dg01 N/A 100 A A RAID10 4 0 512k QTOF 00c0ff25c6e90000b86d8b5a00000000 Disabled 0 512n

Something occurred where there are 2 more drives assigned to this disk-group than the 4 required. A log review ill need to be done to determine which drives should be in the disk-group and which do not belong.

Please open a support case with HPE Support using the following web link, if the unit is under warranty: https://support.hpe.com/hpesc/public/home

If the unit is out of warranty, you can open a chat support case with HPE using the following web link to check for options: https://pg-receiver-pro.glb.itcs.hpe.com/WCLWeb/WCLEntry.aspx

Cheers,
Shawn

I work for Hewlett Packard Enterprise. The comments in this post are my own and do not represent an official reply from HPE. No warranty or guarantees of any kind are expressed in my reply.