topic Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem in MSA Storage

HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Thu, 15 Jul 2021 14:10:32 GMT

Hello , We have an HPE MSA 2040 storage with 15 disks configured in RAID5. the 15th disk is the spare disk initially chosen during configuration. The 4th disk failed and we changed it with a new disk and after changing the disk recovery did not start. We have chosen the new drive as global spare after the change and the rebuild is still not triggered. Please to help us it's really urgent

Capture after replacement

volume_up

content_copy

share

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JonPaul — Thu, 15 Jul 2021 17:00:28 GMT

Is the Disk Group FTOL? (Fault Tolerant)
If the DG was initially built as a 14 drive RAID 5 set and the 15th drive was set as the SPARE then the DG may already be in the process of RCON (Recontruction). The MSA does not utilize 'Slot Affinity', meaning that the DG/RAID set will rebuild on the spare drive wherever it is and not reconstruct onto the same drive that failed, unless that drive is replaced and made a SPARE before the DG starts rebuild on another SPARE drive.
If the DG is still in a degraded state then is the replacement drive a compatible replacement? Same type (SAS, MDL-SAS, SSD) and same or larger capacity?

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Thu, 15 Jul 2021 18:22:52 GMT

Dear JonPaul ,

Thank you for your support !

I'm sorry, I can't load the images here

here is more detail on our problem below. regarding your questions: how to know if the disc and FTOL (Fault Torerant).

When the drive failed we replaced it two weeks later with a drive of the same size and same reference.

I think under normal conditions MSA should rebuild on the 15th disk which is configured as spare. But so far the new disk and the 15th disk still have the status Spare. and it does not blink.

here is an overview of the state of the discs

Here is a preview of one of the working disc

here is the preview of the new disc

We had to do a health check on the MSA here is the result

Scrubbing Enabled

Available Drives, Global Spares, or Dedicated Spares found in array but Disk Background Scrub Not Enabled

Unhealthy Components Chec !

one of the power supply is not connected on power

I remind you that both controllers are up to date. we even restart them:

A>>GL225POOO2-02

B>>GL225POOO2-02

Thanks for considering the captures and getting back to us.

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JonPaul — Thu, 15 Jul 2021 19:24:28 GMT

I am unable to see any of the images.
From the CLI you can see the status of the disk-groups with the command: show disk-groups
Of interest are the 'Class' - should be Virtual or Linear
And 'status' - you can look at the 'help show disk-groups' command to see the options for that field.

You can also get info about the disks from the CLI command: show disks
I'm interested to see if you have 'GLOBAL SP' or a dedicated spare in the 'usage' field
It may be a good test to change the drives from SPARE to AVAIL and create a new SPARE or enable Dynamic Sparing and let that attempt to grab an AVAIL drive.

Also of interest would be the Advanced setting of 'Dynamic Sparing' CLI command: show advanced-settings
Another question, when you removed the faulty drive did you wait 30sec+ before inserting the replacement drive?
I don't think this has anything to do with the reconstruct not starting but can cause an issue with newly inserted drives as a quick replacement may not allow the disk-group to timeout the drive and the system gets confused if the drive is new or not.

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Thu, 15 Jul 2021 19:36:34 GMT

Dear JonPaul,

Thank you for support !

I can't load the images here
. otherwise I was going to share the screenshots with you.

isn't there a way to share the screenshots with you?

We will do the action recommended by you and get back to you.

We didn't wait 30 seconds before inserting the new disc. he was replaced immediately

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Fri, 16 Jul 2021 12:03:18 GMT

Dear JonPaul ,

Thank you for your support !

here is more detail on our problem below. regarding your questions: how to know if the disc and FTOL (Fault Torerant).

When the drive failed we replaced it two weeks later with a drive of the same size and same reference.

I think under normal conditions MSA should rebuild on the 15th disk which is configured as spare. But so far the new disk and the 15th disk still have the status Spare. and it does not blink.

here is an overview of the state of the discs ( the 4th and 15th disc in 'S' condition)

Here is a preview of one of the working disc

here is the preview of the new replacement disc

We had to do a health check on the MSA here is the result

Unhealthy Components Chec !

one of the power supply is not connected on power

I remind you that both controllers are up to date. we even restart them

Thanks for considering the captures and getting back to us.

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JonPaul — Fri, 16 Jul 2021 17:46:33 GMT

I am unable to duplicate the issue you are seeing.
I see that you are using Virtual storage (Disk usage is 'Pool A, Standard Tier') which means you should be using Global SPAREs. I can't determine if that is what you are using as the disk 1.4 image you showed is Usage 'AVAIL'. Although your image of the entire enclosure seems to indicate that it is a SPARE.
The last things I would try before entering a case with HPE support is:
1. Logically remove all SPAREs --> set spares to AVAIL
2. Physically remove the AVAIL drives - keep these out for a couple of minutes
3. Run a manual rescan: rescan disk channels in the SMU
4. install a drive in Slot 4
5. Set drive in Slot 4 as a GLOBAL SPARE
6. Run the manual rescan again
If that does not get the reconstruction to start then a case with HPE Support is needed to review deeper.

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Tue, 27 Jul 2021 11:26:39 GMT

Hello JonPaul,

Thanks for your feedback . we requested a downtime which was pending validation. today we will make your recommendations and get back to you.

Thank you again to you and to the HPE community team

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Thu, 29 Jul 2021 13:25:47 GMT

Dear JonPaul,

Once again thank you for your support.

We took the actions you recommended to resolve the issue. after applying them the problem was not solved ... the disks are still in the spare state and the rebuild did not start.

Please we need your support

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Tue, 03 Aug 2021 16:06:42 GMT

Dear team ,

So far our problem has not yet been resolved. could you help us take it to the next step .. the hpe support warranty has expired .. we need help to avoid any loss given to the next failed drive.

I thank JonPaul for his availability and assistance and I would like him to give me a feedback

Best regards !

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JonPaul — Tue, 03 Aug 2021 16:36:25 GMT

I'm sorry that this issue persists. It appears that the investigation will take more than can be done over the forum and requires direct connection with HPE Support. There are numerous options for getting HPE support after warranty is expired.

Re: HPE STORAGE MSA 2040 , After changing a defective disk, disk rebuilding problem

JBGUIT — Tue, 03 Aug 2021 17:18:37 GMT

Dear JonPaul,

Thank you for your feedback.

Can you share us these support options ?

Best regards !!