Please help, one old 3PAR servicemag start failed after power on

[lj20201029]

Firstly I found pd 37 degraded so I use servicemag start -pdid 37 to replace it, but the cmd failed after running 4 hours.

cli% servicemag status -d 3 0
A servicemag start command failed on this magazine.
The command completed at Mon Nov  2 18:51:40 2020.
The output of the servicemag start was:
servicemag start -pdid 37
... servicing disks in mag: 3 0
...      normal disks:  WWN [5000CCA01790E0F4] Id [36]  diskpos [0] 
....................    WWN [5000CCA01790E6A8] Id [38]  diskpos [2] 
....................    WWN [5000CCA029B66F18] Id [59]  diskpos [3] 
...  not normal disks:  WWN [5000CCA01792282C] Id [37]  diskpos [1] 
... relocating chunklets to spare space...
... chunklet 36:7 - move_error,src_set_invalid, will not move
... chunklet 36:8 - move_error,src_set_invalid, will not move
... chunklet 36:9 - move_error,src_set_invalid, will not move
.......... 
There are  986 failed same as above in PD 36
There are 1242 failed same as above in PD 37
There are 1481 failed same as above in PD 38
There are   8   failed same as above in PD 59

I found the numble of the "move_error,src_invalid, will not move" error of each pd equal to the Normal chunklets -Used - OK value.

3PAR-T400 cli% showpd -c
                                    --------- Normal Chunklets ---------- ----- Spare Chunklets ------
                                    -- Used --- -------- Unused --------- -- Used -- ---- Unused -----
 Id CagePos Type    State     Total     OK Fail  Free Uninit Unavail Fail    OK Fail  Free Uninit Fail
--- 4:8:0   unknown failed        0      0    0     0      0       0    0     0    0     0      0    0
 56 4:8:0?  SSD     degraded    185     67    0   100     18       0    0     0    0     0      0    0
 36 3:0:0   FC      normal     2234    986    0   134    927       0    0     0    0    53    134    0
 37 3:0:1   FC      degraded   2234   1242    0   121    684       0    0     0    0   187      0    0
 38 3:0:2   FC      normal     2234   1481    0   131    436       0    0     0    0    56    130    0
 59 3:0:3   FC      normal     2234      8    0   149   1891       0    0     0    0    68    118    0

And all battary condition is below, so the system is in write through mode.

cli% showbattery
Node PS Bat Assy_Serial -State-- ChrgLvl(%) -ExpDate-- Expired Testing
   0  0   0 446B305F    Degraded        100 05/16/2018 Yes     No     
   0  1   0 501B304B    Degraded        100 06/29/2018 Yes     No     
   1  0   0 443B3036    Degraded        100 04/22/2018 Yes     No     
   1  1   0 502B3016    Degraded        100 07/11/2018 Yes     No     
   2  0   0 446B305F    Degraded        100 05/16/2018 Yes     No     
   2  1   0 501B304B    Degraded        100 06/29/2018 Yes     No     
   3  0   0 443B3036    Degraded        100 04/22/2018 Yes     No     
   3  1   0 502B3016    Degraded        100 07/11/2018 Yes     No

cli%  shownode -verbose
............... node 0,1,2,3 ............
Battery State           :   Degraded       
Battery Detail State    :   Expired,Unknown
Battery Charge State    :   FullyCharged   
Battery Charge Level(%) :   100            
Max Battery Life(mins)  :   25

 Question:

1. How to continue replace the failed PD 37?

2.How to deal with the two 4:8:0 PDs?

3.Is it OK to use the storage in write through mode? (I just use it in testing, not in production environment)

Any suggestion is welcome.

[sbhat09]

Hello,

Can you please provide the below information?

showpd 18
showpd 56

showpd -i 18
showpd -i 56

showpd -i -p -cg # -mg #

(Replace # with cage id and mag id)

showpd -c -degraded

Regards,

Srinivas

[lj20201029]

Sure，I delete the useless lines.

3PAR-T400 cli% showpd
                                 -----Size(MB)----- ----Ports----             
 Id CagePos Type    RPM State        Total     Free A      B      Capacity(GB)

 18 1:1:2   FC       15 normal      571904    13312 0:0:2* 1:0:2           600
56 4:8:0?  SSD     150 degraded     47360    30208 -----  -----            50

--- 4:8:0   unknown   0 failed           0        0 2:0:1- 3:0:1-            0
 36 3:0:0   FC       15 normal      571904   271616 0:0:4* 1:0:4           600
 37 3:0:1   FC       15 degraded    571904   206080 0:0:4- 1:0:4-          600
 38 3:0:2   FC       15 normal      571904   145152 0:0:4* 1:0:4           600
 39 0:2:3   FC       15 normal      571904    61696 0:0:1  1:0:1*          600
 59 3:0:3   FC       15 normal      571904   522240 0:0:4  1:0:4*          600

------------------------------------------------------

 cli% showpd -i
 Id CagePos State    ----Node_WWN---- --MFR-- -----Model------ ----Serial---- -FW_Rev-- Protocol MediaType
 18 1:1:2   normal   5000CCA01792832C HITACHI HVIPC0600GBFC15K JZXLKW5J       3P03      FC       Magnetic 
  56 4:8:0?  degraded 5000A72030046E48 STEC    M8IOPS-50        STM00013C7C3   2851,1612 SATA     --       

--- 4:8:0   failed   208F0002AC008B3C                                                   --       --       
 36 3:0:0   normal   5000CCA01790E0F4 HITACHI HVIPC0600GBFC15K JZXKP0SJ       3P03      FC       Magnetic 
 37 3:0:1   degraded 5000CCA01792282C HITACHI HVIPC0600GBFC15K JZXLBU9J       3P03      FC       Magnetic 
 38 3:0:2   normal   5000CCA01790E6A8 HITACHI HVIPC0600GBFC15K JZXKPDJJ       3P03      FC       Magnetic 
 39 0:2:3   normal   2000B45253C99B63 SEAGATE SEGLE0600GBFC15K 6SL9YJH1       3P00      FC       Magnetic 
 59 3:0:3   normal   5000CCA029B66F18 HITACHI HVIPC0600GBFC15K LXY7A99N       3P03      FC       Magnetic 

------------------------------------------------------------------------------------------------------------------------------------
showpd -i -p -cg 3 -mg 0 should be:
 36 3:0:0   normal   5000CCA01790E0F4 HITACHI HVIPC0600GBFC15K JZXKP0SJ       3P03      FC       Magnetic 
 37 3:0:1   degraded 5000CCA01792282C HITACHI HVIPC0600GBFC15K JZXLBU9J       3P03      FC       Magnetic 
 38 3:0:2   normal   5000CCA01790E6A8 HITACHI HVIPC0600GBFC15K JZXKPDJJ       3P03      FC       Magnetic 
 59 3:0:3   normal   5000CCA029B66F18 HITACHI HVIPC0600GBFC15K LXY7A99N       3P03      FC       Magnetic 

------------------------------------------------------------------------------------------------------------------------------------

 cli% showpd -c -failed -degraded
                                   -------- Normal Chunklets --------- ---- Spare Chunklets ----
                                   -- Used -- -------- Unused -------- - Used - ---- Unused ----
 Id CagePos Type    State    Total    OK Fail Free Uninit Unavail Fail  OK Fail Free Uninit Fail
--- 4:8:0   unknown failed       0     0    0    0      0       0    0   0    0    0      0    0
  0 0:0:0   FC      degraded  2234  1231    0  107    709       0    0   0    0  187      0    0
 37 3:0:1   FC      degraded  2234  1242    0  121    684       0    0   0    0  187      0    0
 56 4:8:0?  SSD     degraded   185    67    0  100     18       0    0   0    0    0      0    0

[sbhat09]

Hello,

For PD id 37,

cgcmd -tc tty -C cage3 -c "poweroffdrive 1" -p a

cgcmd -tc tty -C cage3 -c "powerondrive 1" -p a     

And then,

controlmag onloop cage # mag #  

(Replace # with cage id and mag id)

I am still not clear about PD 56.

Have you replaced the drive with the same drive type?

Please share these command outputs:

showpd -i -p -cg 4 -mg 8

showpd -s 56

showpd -s 37

Regards,

Srinivas

[louisji2008]

maybe you  forgot delete the ghost pd?