Odd behavior upon replacing failed drives in 3PAR 8200

[phodges]

Hello,  I have multiple failed disks on my 3PAR that I am trying to replace.  When I put in the new drive, the servicemag resumes, and appears to be working correctly, but it stops with a failed state.  a servicemag status -d shows the below output for about 10 chunklets:

chunklet 8:1662 - move_error,duplicate_dest, failed move

Only about 700 or so chunklets out of the original 2200 marked to move will move over.

When I view the disk from the service console, it appears very odd 

Free 0 GiB

Volumes 715 GiB

Spares 0 GiB

Unavailable 1,920 GiB

Failed 0 GiB

Total drive capacity 1,675 GiB

Support tried to issue the following commands: 

servicemag unmark

servicemag clearstatus

servicemag resume

it retried, and after 101 retries, finally failed again.

I'm about to call support again, but any help would be greatly appreciated.

[Satish04]

Hi Phodges,

Hope all the drives are compatible with the Storage system OS.

I would suggest you check the reason for the drive failure using the "servicemag status -d" command and also check PD ID's are unique.
Could you please share the output of "servicemag status -d" and "showpd -s" command

Regards,
Satish

[veeyarvi]

Hi Phodges,

The servicemag failure with duplicate_destination should be related to missing a critical patch in the system. Could you confirm the array is running on 3.2.2 MU6 (plus all critical patches)? If not, the support might perform a few manual steps to resolve the issue (if OS upgrade not immediately) possible.

Regards,

Veeyaarvi

[phodges]

@veeyarvi 

Thank you for your reply.  Yes, these are old spare drives that we had.  They are the same model number and part number as the failed drives.

I am currently being helped by a L2 technician who was able to get it working manually.  

I'm waiting to hear back from him today, because I'm wondering if the issue was caused by the following:

Upon discovery of the amber indicator on the failed drives, a coworker pulled the failed drive out and put it back in, so the indicator light turned back green, but the drive still showed as failed in the system.  I wonder if because of this, there was a further step that needed to happen before replacing the failed drive with a good drive?

[veeyarvi]

Hi Phodes

Do you remember what it was showing with 'showpd -s" for that particular drive? What was the servicemag status? Any idea?

Normally, if a drive marked 'failed' by system, a simple re-seat will not change that status. So, I assume, that (re-seat) had no effect on this behaviour.

Regards, 
Veeyaarvi