topic Re: 3PAR. Try to change disk in HPE 3PAR StoreServ Storage

3PAR. Try to change disk

Livitin — Sat, 02 May 2015 06:10:43 GMT

Hi.

3par-7200 cli% showversion
Release version 3.1.2 (MU3)
Patches: P18

Component Name Version
CLI Server 3.1.2 (MU3)
CLI Client 3.1.2 (MU3)
System Manager 3.1.2 (MU3)
Kernel 3.1.2 (MU3)
TPD Kernel Code 3.1.2 (MU3)

We found failed disk and successfully changed it to service disk that we have. Later we tried to change service disk to the brand new disk. We have problem.

We don't have spare disks in the system.

1. We losted disk #13 (1.1.0)

2. We insert new service disk and it's number was #24 and it was saccessfuly started.

3. We bought brand new disk, we removed service disk and insert new disk. The number of new disk is #13

Ho to tell to system to change disks?

Now we have in the 3par console:

1. #13 cage1 -state = degraded, -State description = Missing, servicing

2. #24 cage1 -state = Failed, -State description = Vacated Prolonged Missing, No Valid Ports, Servicing

If i try servicemag status -d:

Cage 1, magazine 1:
A servicemag start command failed on this magazine.
The command completed at Sat May 2 09:12:42 2015.
The output of the servicemag start was:
Failed --
Unable to run servicemag Start command. Servicemag is either active or has failed for this cage and magazine.
Please rectify any error conditions and issue "servicemag unmark" before retrying.

servicemag start -pdid 24 -- Failed

Re: 3PAR. Try to change disk

Dennis Handly — Sat, 02 May 2015 11:25:33 GMT

>We found failed disk and successfully changed it to service disk that we have.

What do you mean by service disk, a spare? Is it the same size/speed/type as the failed one?

What exactly did you do? Why do you say "successfully"?

>Later we tried to change service disk to the brand new disk.

You can't. Once you put the disk in, it usually stays in until it fails. Unless you use servicemag.

>We don't have spare disks in the system.

Why would you have spares? You should have spare chunklets.

>1. We lost disk #13 (1.1.0)

Then you use servicemag to replace it.

>2. We insert new service disk and it's number was #24 and it was successfully started.

That does you no good, that disk is completely separate from #13 and unless you can dismisspd, it's part of the system.

>3. We bought brand new disk, we removed service disk and insert new disk. The number of new disk is #13

How did you "remove" #24?

>How to tell to system to change disks?

Only by using servicemag.

>Now we have in the 3PAR console:

1. #13 cage1 -state = degraded, -State description = Missing, servicing

2. #24 cage1 -state = Failed, -State description = Vacated Prolonged Missing, No Valid Ports, Servicing

Please paste or attach the following info using the CLI:

showpd -i

showpd -s

Also include the exact commands and what you did.

>If i try servicemag status -d:

Did you try servicemag before?

>servicemag start -pdid 24 -- Failed

When you use servicemag, you must replace the disk, you can't just remove it.

To remove a disk you must use dismisspd, which is very hard to do.

Re: 3PAR. Try to change disk

Livitin — Sat, 02 May 2015 16:44:44 GMT

3par-7200 cli% servicemag status -d
Cage 1, magazine 1:
A servicemag start command failed on this magazine.
The command completed at Sat May 2 09:12:42 2015.
The output of the servicemag start was:
Failed --
Unable to run servicemag Start command. Servicemag is either active or has failed for this cage and magazine.
Please rectify any error conditions and issue "servicemag unmark" before retrying.

servicemag start -pdid 24 -- Failed
3par-7200 cli% showpd -i
Id CagePos State    ----Node_WWN---- --MFR-- -----Model------ -Serial- -FW_Rev- Protocol MediaType
0 0:0:0   normal   5000C5005EB4224C SEAGATE SYJKT0300GBAS15K 6XN2QDAS 3P01     SAS      Magnetic
1 0:1:0   normal   5000C5005EB422C4 SEAGATE SYJKT0300GBAS15K 6XN2QTMF 3P01     SAS      Magnetic
2 0:2:0   normal   5000C5005EAFE25C SEAGATE SYJKT0300GBAS15K 6XN2QGN7 3P01     SAS      Magnetic
3 0:3:0   normal   5000C5005F072A48 SEAGATE SYJKT0300GBAS15K 6XN2VRFY 3P01     SAS      Magnetic
4 0:4:0   normal   5000C5005EB0F8B4 SEAGATE SYJKT0300GBAS15K 6XN2QDND 3P01     SAS      Magnetic
5 0:5:0   normal   5000C5005F0748B4 SEAGATE SYJKT0300GBAS15K 6XN2VR9G 3P01     SAS      Magnetic
6 0:6:0   normal   5000C5005EB04BF8 SEAGATE SYJKT0300GBAS15K 6XN2PX4L 3P01     SAS      Magnetic
7 0:7:0   normal   5000C5005EB2D8C4 SEAGATE SYJKT0300GBAS15K 6XN2Q6RB 3P01     SAS      Magnetic
8 0:8:0   normal   5000C5005F20A598 SEAGATE SYJKT0300GBAS15K 6XN315QR 3P01     SAS      Magnetic
9 0:9:0   normal   5000C5005EB0C334 SEAGATE SYJKT0300GBAS15K 6XN2Q1V3 3P01     SAS      Magnetic
10 0:10:0 normal   5000C5005EB0B7B4 SEAGATE SYJKT0300GBAS15K 6XN2QEMJ 3P01     SAS      Magnetic
11 0:11:0 normal   5000C5005EFABB60 SEAGATE SYJKT0300GBAS15K 6XN2W9QA 3P01     SAS      Magnetic
12 1:0:0   normal   5000CCA01B9400DB HITACHI HMRSK2000GBAS07K YFJMD94D 3P00     SAS      Magnetic
13 1:1:0? degraded 5000CCA01BC09447 HITACHI HMRSK2000GBAS07K YFKDX86K 3P00     SAS      Magnetic
14 1:2:0   normal   5000CCA01C931303 HITACHI HMRSK2000GBAS07K YGJLWG3D 3P00     SAS      Magnetic
15 1:3:0   normal   5000CCA01C92BCAB HITACHI HMRSK2000GBAS07K YGJLPPVD 3P00     SAS      Magnetic
16 1:4:0   normal   5000CCA01B9400AF HITACHI HMRSK2000GBAS07K YFJMD8TD 3P00     SAS      Magnetic
17 1:5:0   normal   5000CCA01B93FB27 HITACHI HMRSK2000GBAS07K YFJMBXBD 3P00     SAS      Magnetic
18 1:6:0   normal   5000CCA01B940023 HITACHI HMRSK2000GBAS07K YFJMD7ND 3P00     SAS      Magnetic
19 1:7:0   normal   5000CCA01C898023 HITACHI HMRSK2000GBAS07K YGJEM77D 3P00     SAS      Magnetic
20 1:8:0   normal   5000CCA01B9214D7 HITACHI HMRSK2000GBAS07K YFJLAJBD 3P00     SAS      Magnetic
21 1:9:0   normal   5000CCA01C926813 HITACHI HMRSK2000GBAS07K YGJLJ26D 3P00     SAS      Magnetic
22 1:10:0 normal   5000CCA01C8E5BEF HITACHI HMRSK2000GBAS07K YGJJ922D 3P00     SAS      Magnetic
23 1:11:0 normal   5000CCA01C92B427 HITACHI HMRSK2000GBAS07K YGJLP48D 3P00     SAS      Magnetic
24 1:1:0? failed   5000CCA01C996FFF HITACHI HMRSK2000GBAS07K YGJRBY5K 3P00     SAS      Magnetic
--------------------------------------------------------------------------------------------------
25 total
3par-7200 cli% showpd -s
Id CagePos Type -State-- ------------------Detailed_State------------------
0 0:0:0   FC   normal   normal
1 0:1:0   FC   normal   normal
2 0:2:0   FC   normal   normal
3 0:3:0   FC   normal   normal
4 0:4:0   FC   normal   normal
5 0:5:0   FC   normal   normal
6 0:6:0   FC   normal   normal
7 0:7:0   FC   normal   normal
8 0:8:0   FC   normal   normal
9 0:9:0   FC   normal   normal
10 0:10:0 FC   normal   normal
11 0:11:0 FC   normal   normal
12 1:0:0   NL   normal   normal
13 1:1:0? NL   degraded missing,servicing
14 1:2:0   NL   normal   normal
15 1:3:0   NL   normal   normal
16 1:4:0   NL   normal   normal
17 1:5:0   NL   normal   normal
18 1:6:0   NL   normal   normal
19 1:7:0   NL   normal   normal
20 1:8:0   NL   normal   normal
21 1:9:0   NL   normal   normal
22 1:10:0 NL   normal   normal
23 1:11:0 NL   normal   normal
24 1:1:0? NL   failed   vacated,prolonged_missing,no_valid_ports,servicing
---------------------------------------------------------------------------
25 total

Re: 3PAR. Try to change disk

Livitin — Sat, 02 May 2015 16:47:09 GMT

We can return disk #24 to it's position. Then we need to try remove it in right way. Am I right?

Re: 3PAR. Try to change disk

Livitin — Sat, 02 May 2015 22:02:56 GMT

So I need :

return #24 to its place.
dismisspd 24
remove disk from the system
insert new disk (#13)
what I need to do to return system to normal state?

Re: 3PAR. Try to change disk

Dennis Handly — Sun, 03 May 2015 00:38:51 GMT

13 1:1:0? degraded 5000CCA01BC09447 HITACHI HMRSK2000GBAS07K YFKDX86K 3P00 SAS Magnetic

24 1:1:0? failed 5000CCA01C996FFF HITACHI HMRSK2000GBAS07K YGJRBY5K 3P00 SAS Magnetic

13 1:1:0? NL degraded missing,servicing

24 1:1:0? NL failed vacated,prolonged_missing,no_valid_ports,servicing

These match what you said. Unfortunately you put the disk in the same slot.

>1. return #24 to its place.

Yes

>2. dismisspd 24

It may not be that easy, depending on whether data or spares were added to it.

>3. remove disk from the system

Yes

>4. insert new disk (#13)

Then the "servicemag resume".

>5. what I need to do to return system to normal state?

Unfortunately it requires analysis after each step. You may want to get Support involved.

Especially since you put that other disk in the same slot.

Re: 3PAR. Try to change disk

Livitin — Sun, 03 May 2015 05:25:40 GMT

1. step - we were insert #24 disk:

3par-7200 cli% showpd -i
Id CagePos State    ----Node_WWN---- --MFR-- -----Model------ -Serial- -FW_Rev- Protocol MediaType
0 0:0:0   normal   5000C5005EB4224C SEAGATE SYJKT0300GBAS15K 6XN2QDAS 3P01     SAS      Magnetic
1 0:1:0   normal   5000C5005EB422C4 SEAGATE SYJKT0300GBAS15K 6XN2QTMF 3P01     SAS      Magnetic
2 0:2:0   normal   5000C5005EAFE25C SEAGATE SYJKT0300GBAS15K 6XN2QGN7 3P01     SAS      Magnetic
3 0:3:0   normal   5000C5005F072A48 SEAGATE SYJKT0300GBAS15K 6XN2VRFY 3P01     SAS      Magnetic
4 0:4:0   normal   5000C5005EB0F8B4 SEAGATE SYJKT0300GBAS15K 6XN2QDND 3P01     SAS      Magnetic
5 0:5:0   normal   5000C5005F0748B4 SEAGATE SYJKT0300GBAS15K 6XN2VR9G 3P01     SAS      Magnetic
6 0:6:0   normal   5000C5005EB04BF8 SEAGATE SYJKT0300GBAS15K 6XN2PX4L 3P01     SAS      Magnetic
7 0:7:0   normal   5000C5005EB2D8C4 SEAGATE SYJKT0300GBAS15K 6XN2Q6RB 3P01     SAS      Magnetic
8 0:8:0   normal   5000C5005F20A598 SEAGATE SYJKT0300GBAS15K 6XN315QR 3P01     SAS      Magnetic
9 0:9:0   normal   5000C5005EB0C334 SEAGATE SYJKT0300GBAS15K 6XN2Q1V3 3P01     SAS      Magnetic
10 0:10:0 normal   5000C5005EB0B7B4 SEAGATE SYJKT0300GBAS15K 6XN2QEMJ 3P01     SAS      Magnetic
11 0:11:0 normal   5000C5005EFABB60 SEAGATE SYJKT0300GBAS15K 6XN2W9QA 3P01     SAS      Magnetic
12 1:0:0   normal   5000CCA01B9400DB HITACHI HMRSK2000GBAS07K YFJMD94D 3P00     SAS      Magnetic
13 1:1:0? degraded 5000CCA01BC09447 HITACHI HMRSK2000GBAS07K YFKDX86K 3P00     SAS      Magnetic
14 1:2:0   normal   5000CCA01C931303 HITACHI HMRSK2000GBAS07K YGJLWG3D 3P00     SAS      Magnetic
15 1:3:0   normal   5000CCA01C92BCAB HITACHI HMRSK2000GBAS07K YGJLPPVD 3P00     SAS      Magnetic
16 1:4:0   normal   5000CCA01B9400AF HITACHI HMRSK2000GBAS07K YFJMD8TD 3P00     SAS      Magnetic
17 1:5:0   normal   5000CCA01B93FB27 HITACHI HMRSK2000GBAS07K YFJMBXBD 3P00     SAS      Magnetic
18 1:6:0   normal   5000CCA01B940023 HITACHI HMRSK2000GBAS07K YFJMD7ND 3P00     SAS      Magnetic
19 1:7:0   normal   5000CCA01C898023 HITACHI HMRSK2000GBAS07K YGJEM77D 3P00     SAS      Magnetic
20 1:8:0   normal   5000CCA01B9214D7 HITACHI HMRSK2000GBAS07K YFJLAJBD 3P00     SAS      Magnetic
21 1:9:0   normal   5000CCA01C926813 HITACHI HMRSK2000GBAS07K YGJLJ26D 3P00     SAS      Magnetic
22 1:10:0 normal   5000CCA01C8E5BEF HITACHI HMRSK2000GBAS07K YGJJ922D 3P00     SAS      Magnetic
23 1:11:0 normal   5000CCA01C92B427 HITACHI HMRSK2000GBAS07K YGJLP48D 3P00     SAS      Magnetic
24 1:1:0   failed   5000CCA01C996FFF HITACHI HMRSK2000GBAS07K YGJRBY5K 3P00     SAS      Magnetic
--------------------------------------------------------------------------------------------------
25 total

3par-7200 cli% showpd -s
Id CagePos Type -State-- -Detailed_State--
0 0:0:0   FC   normal   normal
1 0:1:0   FC   normal   normal
2 0:2:0   FC   normal   normal
3 0:3:0   FC   normal   normal
4 0:4:0   FC   normal   normal
5 0:5:0   FC   normal   normal
6 0:6:0   FC   normal   normal
7 0:7:0   FC   normal   normal
8 0:8:0   FC   normal   normal
9 0:9:0   FC   normal   normal
10 0:10:0 FC   normal   normal
11 0:11:0 FC   normal   normal
12 1:0:0   NL   normal   normal
13 1:1:0? NL   degraded missing,servicing
14 1:2:0   NL   normal   normal
15 1:3:0   NL   normal   normal
16 1:4:0   NL   normal   normal
17 1:5:0   NL   normal   normal
18 1:6:0   NL   normal   normal
19 1:7:0   NL   normal   normal
20 1:8:0   NL   normal   normal
21 1:9:0   NL   normal   normal
22 1:10:0 NL   normal   normal
23 1:11:0 NL   normal   normal
24 1:1:0   NL   failed   vacated,servicing
------------------------------------------
25 total

3par-7200 cli% servicemag resume 1 1.0
Are you sure you want to run servicemag?
select q=quit y=yes n=no: y
error: incorrect arguments
servicemag resume -- Failed
Command failed

servicemag start -pdid 24 -- Failed

Do I need to run servicemag unmark 1 1.0 now ?

Re: 3PAR. Try to change disk

Livitin — Sun, 03 May 2015 05:29:50 GMT

3par-7200 cli% dismisspd 24
Error : Pd id 24 has 256 chunklet marked as spare

Re: 3PAR. Try to change disk

Dennis Handly — Sun, 03 May 2015 07:14:30 GMT

>Error : Pd id 24 has 256 chunklet marked as spare

As I said, this is hard.

You must do a sequence of commands:

removespare -f 24:a

Now try dismisspd 24 again. If there are still used chunklets, you'll have to write a script to move them off.

showpdch 24

Re: 3PAR. Try to change disk

Livitin — Sun, 03 May 2015 10:11:37 GMT

3par-7200 cli% removespare -f 24:a
256 spares removed
3par-7200 cli% dismisspd 24
Error : Pd id 24 is referenced by chunklet 12:1549

Now I have to remove this chinklet?

Re: 3PAR. Try to change disk

Livitin — Sun, 03 May 2015 10:24:57 GMT

How to move referenced chinklets?

3par-7200 cli% showpdch 24
No chunklet information available.
3par-7200 cli% dismisspd 24
Error : Pd id 24 is referenced by chunklet 12:1550

Re: 3PAR. Try to change disk

Dennis Handly — Mon, 04 May 2015 12:33:49 GMT

>% dismisspd 24
>Error : Pd id 24 is referenced by chunklet 12:1549

>Now I have to remove this chunklet?

You have to move it. You can't remove it, it has data.

>How to move referenced chunklets?

This is the hard part, you need to do scripting. What does this show:

showpdch -from 24

And might as well:

showpdch -from 13

Re: 3PAR. Try to change disk

Livitin — Mon, 04 May 2015 12:57:36 GMT

Grate thank's to Dennis Handly!

I did it.

So, full solution:

For existing disk need to execute commands

to remove spare: removespare -f pdid:a
to move existing chinklets first create script with command: movech -perm -ovrd -f pdid:chinkletnumber (you can collect pdid and chinkletnumber from showpdch -from pdid. I used Exel to create 722 commands)
remove disk from the system: dismisspd pdid

Insert a new disk then run servicemag resume cage_ID magazine_ID

Re: 3PAR. Try to change disk

Dennis Handly — Mon, 04 May 2015 13:11:11 GMT

>to move existing chinklets first create script with command: movech -perm -ovrd -f pdid:chunkletnumber (you can collect pdid and chunkletnumber from showpdch -from pdid. I used Excel to create 722 commands)

Well, you could write a one liner bash script to do this.

Re: 3PAR. Try to change disk

Jozef_Novak — Thu, 14 May 2015 07:06:14 GMT

Hello Dennis,

wouldn't a movepd do the job here ? (not sure, just asking).

Thanks,

Re: 3PAR. Try to change disk

Dennis Handly — Fri, 15 May 2015 07:59:35 GMT

>wouldn't a movepd do the job here?

One would think but typically there isn't enough spares in the right places for the system to do that.

It errors out since there isn't a -ovrd option.

Re: 3PAR. Try to change disk

ChandraSama — Mon, 18 Apr 2016 08:37:30 GMT

could you please help me with the one liner bash script to accomplish this task? I have 2127 chunklets to be moved. Appreciate you help.

Re: 3PAR. Try to change disk

MikeGolbin — Thu, 29 Jun 2017 16:00:17 GMT

In windows command line you can make script this way (pd_chunklets.txt is the output of the shodpdch # without header and footer) :

for /f "tokens=1,2" %i in (pd_chunklets.txt) do echo movech -f -perm -ovrd %i:%j >>movech.txt

and then paste content of the movech.txt into putty window