topic Re: cannot replace a disk in 3par 7200 in HPE 3PAR StoreServ Storage

cannot replace a disk in 3par 7200

Yavor — Tue, 07 May 2019 07:48:46 GMT

Hello,

we have a 7200 running 3.2.1

tried to replace the disk 14 and it did not get included in the system.

Afterwards disk 9 failed too.

---------------------------------------------------------------------------------------------------------------------------------

servicemag status -d
Cage 0, magazine 9:
The magazine was successfully brought offline by a servicemag start command.
The command completed at Thu Apr 25 21:25:00 2019.
The output of the servicemag start was:
servicemag start -wait -pdid 9
... servicing disks in mag: 0 9
... normal disks:
... not normal disks: WWN [5000C50076A6FCFC] Id [ 9]
... relocating chunklets to spare space...
... bypassed mag 0 9
Failed --
failed to turn drive's LED amber
servicemag start -wait -pdid 9 -- Succeeded

Cage 0, magazine 14:
The magazine was successfully brought offline by a servicemag start command.
The command completed at Wed Apr 24 22:40:04 2019.
The output of the servicemag start was:
servicemag start -wait -pdid 14
... servicing disks in mag: 0 14
... normal disks:
... not normal disks: WWN [5000C50072629294] Id [14]
... relocating chunklets to spare space...
... bypassed mag 0 14
Failed --
failed to turn drive's LED amber
servicemag start -wait -pdid 14 -- Succeeded

---------------------------------------------------------------------------------------------------------------------------------

servicemag resume -partial 0 14
Are you sure you want to run servicemag?
select q=quit y=yes n=no: y
Failed --
Cage 0 mag 14 'servicemag start' was started since Wed Apr 24 22:39:54 2019 or it has been interrupted. Please run 'servicemag status -d' for further details
servicemag resume -partial 0 14 -- Failed

---------------------------------------------------------------------------------------------------------------------------------

showpd -space 14
-----------------(MB)------------------
Id CagePos Type -State- Size Volume Spare Free Unavail Failed
14 0:14:0? FC failed 417792 0 0 0 0 417792
---------------------------------------------------------------
1 total 417792 0 0 0 0 417792

---------------------------------------------------------------------------------------------------------------------------------

showalert -d
Id : 154
State : New
Message Code: 0x00600fa
Repeat Count: Occurred 2 times, first at 2019-04-25 21:24:49 CEST
Time : 2019-04-25 21:24:49 CEST
Severity : Major
Type : Component state change
Component : sw_cage_sled:0:9:0,sw_pd:9
Message : Magazine 0:9:0, Physical Disk 9 Failed (Replace Drive {0x46}, Vacated {0x45}, Prolonged Missing {0xa0})

Id : 153
State : New
Message Code: 0x00600fa
Repeat Count: Occurred 3 times, first at 2019-04-24 22:39:53 CEST
Time : 2019-04-26 17:22:00 CEST
Severity : Major
Type : Component state change
Component : sw_cage_sled:0:14:0,sw_pd:14
Message : Magazine 0:14:0, Physical Disk 14 Failed (Vacated {0x45}, Prolonged Missing {0xa0}, Servicing {0x12})

2 alerts

admitpd returns - addmitted 0 out of 1 disk

Can somebody please advise?

Thank you.

Re: cannot replace a disk in 3par 7200

veeyarvi — Wed, 08 May 2019 10:59:08 GMT

Hello Yavor,

What I understand is the new disk is not recognised by the system. This could be due to the old inform OS version running (3.2.1) since new Drive models require later Inform OS versions (or specific patches with older Inform OS versions).

Re: cannot replace a disk in 3par 7200

Yavor — Wed, 08 May 2019 21:01:53 GMT

Hello again,

@ veeyarvi- thank you for your suggestion.

Disks with the same model already are present in the storage system:

showpd -i
Id CagePos State ----Node_WWN---- --MFR-- -----Model------ -Serial- -FW_Rev- Protocol MediaType -----AdmissionTime------
--- 0:14:0 new 5000CCA01656374F HITACHI HCBRE0450GBAS10K KMHJDWKF 3P02 SAS Magnetic -----------------------
14 0:14:0? failed 5000C50072629294 SEAGATE SLTN0450S5xnN010 S0L0529P 3P01 SAS Magnetic 2014-12-01 17:04:21 CET
75 4:8:0 normal 5000CCA0165620CF HITACHI HCBRE0450GBAS10K KMHJBD3F 3P02 SAS Magnetic 2018-11-08 15:20:46 CET
80 0:19:0 normal 5000CCA0165614AF HITACHI HCBRE0450GBAS10K KMHJAL2F 3P02 SAS Magnetic 2019-03-06 14:40:23 CET
...

I am interested what could cause errors like:

... relocating chunklets to spare space...
... bypassed mag 0 14
Failed --
failed to turn drive's LED amber
servicemag start -wait -pdid 14 -- Succeeded

And in reality no LED was lit on the drive.

We tried returning the old drive to its place, dismisspd, which did not work due to the spare chunklets on other drives referencing the drive 14. So we moved back all the chunklets with movech -perm -ovrd X:Y, and moved them away with movech -perm -ovrd 14:Y. Replaced the drive and it is now admitted with normal status. However the servicemag is still at its old state, so i suppose we'll need to servicemag unmark, servicemag clearstatus.

I am afraid we will hit the same stone once we get the replacement drive for 0 9. I has also no lighs on and the same error in the servicemag status. And the upgrade to 3.3.1 is scheduled for sometime in June.

Thank you.

Re: cannot replace a disk in 3par 7200

veeyarvi — Thu, 09 May 2019 07:52:50 GMT

Hello Yavor,

The message "failed to turn drive's LED amber" normally indicates the drive is completely dead (and hence it did not respond to the LED status change request). I understand the drive had no LED - so it was a confirmed complete failure.

And need to unmark and clearstatus the servicemag to clear the servicemag status for the PD.

Re: cannot replace a disk in 3par 7200

Yavor — Fri, 10 May 2019 07:48:39 GMT

Hello,

thank you veeyarvi for the responce.

What do you make of: ... bypassed mag 0 14

How should one interpret such a message?

Thank you.

Re: cannot replace a disk in 3par 7200

Torsten. — Fri, 10 May 2019 08:04:09 GMT

The disk is set offline then.

Re: cannot replace a disk in 3par 7200

veeyarvi — Fri, 10 May 2019 08:23:58 GMT

Hello Yavor,

The PD is unresponsive and so bypassing is the only action.

Re: cannot replace a disk in 3par 7200

Yavor — Fri, 10 May 2019 15:35:51 GMT

Hello veeyaravi,

do i get this right by interpretting it:
some of the chunklets from the failed disk could not be copyed to spare chunklets, the disk gets bypassed, and the remaining chunklets are reconstructed by the RAID to spares?

Thank you.

Re: cannot replace a disk in 3par 7200

veeyarvi — Fri, 10 May 2019 15:51:07 GMT

Hello Yavor,

Yes, you are right. And in this case, i beleive all chunklets got reconstructed since the disk was completely failed.