Using LVM mirroring to migrate from array to array

[No idea what I'm doing]

Hey, guys. I am working on migrating a bunch of volume groups from an XP1024 onto an XP12K using LVM mirroring. I'm able to create the mirrors without any problem so that my data is now on both arrays. I'm having trouble figuring out the best way to remove the old array from the volume group.

I used physical volume groups to do the mirror. I made a new PVG when I extended the volume group that contained all of the new disks that I presented to the host from the XP12K. I then lvextended each of the LVs in the VG with mirror copy = 1 and specified the new PVG.

My problem is that I can't lvreduce out a physical volume group. It would be easy to remove the XP1024 disks if that was the case. Please help me with the procedure to reduce out these disks:

# vgdisplay -v v42

...

--- Physical volume groups ---
PVG Name vg42_pri
PV Name /dev/dsk/c36t0d0
PV Name /dev/dsk/c36t0d1
PV Name /dev/dsk/c36t0d2
PV Name /dev/dsk/c36t0d3
PV Name /dev/dsk/c36t0d4
PV Name /dev/dsk/c36t0d5
PV Name /dev/dsk/c36t0d6
PV Name /dev/dsk/c36t0d7
PV Name /dev/dsk/c36t1d0
PV Name /dev/dsk/c36t1d1
PV Name /dev/dsk/c36t1d2
PV Name /dev/dsk/c36t1d3

PVG Name vg42_xp1024
PV Name /dev/dsk/c32t0d2
PV Name /dev/dsk/c32t0d4
PV Name /dev/dsk/c32t0d6
PV Name /dev/dsk/c32t1d0
PV Name /dev/dsk/c32t1d2
PV Name /dev/dsk/c32t1d4
PV Name /dev/dsk/c32t0d3
PV Name /dev/dsk/c32t0d5
PV Name /dev/dsk/c32t0d7
PV Name /dev/dsk/c32t1d1
PV Name /dev/dsk/c32t1d3
PV Name /dev/dsk/c32t1d5

----------------------------------------------

casjoh2@baclaw:/home/casjoh2# lvdisplay /dev/vg42/lvol1
--- Logical volumes ---
LV Name /dev/vg42/lvol1
VG Name /dev/vg42
LV Permission read/write
LV Status available/syncd
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 388000
Current LE 24250
Allocated PE 48500
Stripes 0
Stripe Size (Kbytes) 0
Bad block NONE
Allocation PVG-strict/distributed
IO Timeout (Seconds) default



vg42_pri is the PVG containing the XP12K disks.
vg42_xp1024 is the PVG containing the XP1024 disks that I want to remove. There are three LVMs in this VG that I need to take care of and right now all three of them are mirrored between the two PVGs.

Thanks!

[James R. Ferguson]

Hi:

Since LVM mirroring occurs at the logical volume level, you need to work there. You can do:

# lvreduce -m 0 lv_path pv_path

...to target a specific physical volume.

Regards!

...JRF...

[No idea what I'm doing]

Thank you for the response. I believed I tried this the first time around.

I began reducing out each of the PV Names but once I was done with the first one, the mirror was broken. And I then got an error message telling me that the mirror was no longer established when I started reducing the second PV Name. Do I need to remove the
"-m 0" option after the mirror is broke with the first lvreduce command?

[Mel Burslan]

No, actually you need to use -m 0 switch on all logical volumes that have been mirrored on the drive that you are removing. But looking at the number of the disks you are migration out of, I see that it is very easy to make a typographical error, especially if you have an ample amount of logical volumes.

You can querry the pv's with

pvdisplay /dev/dsk/cXtXdX

command to find out which ones have attachments to which logical volumes so that remove these disks from the proper logical volume mirrors.

Hope this helps

[James R. Ferguson · Accepted solution]

Hi:

If any of the physical volumes are alternate links (pvlinks) your first reduction of the mirror copies to zero whould yield a subsequent error:

lvreduce: "MirrorCopies" parameter "0" is not smaller than existing number "0";
therefore no mirrors are removed.

Using Mel's suggestion, you might do :

# cat ./zappvmirrors
#!/usr/bin/sh
typeset VG=vg42
for PV in c32t0d2 c32t0d4 c32t0d6 c32t1d0 \
c32t1d2 c32t1d4 c32t0d3 c32t0d5 \
c32t0d7 c32t1d1 c32t1d3 c32t1d5
do
for LV in lvol1 lvol2 lvol3
do
pvdisplay -v /dev/dsk/${PV} 2> /dev/null | grep -q /dev/${VG}/${LV} && \
lvreduce -m 0 /dev/${VG}/${LV} /dev/dsk/${PV} || echo "skipped ${PV}:${LV}"
done
done

Regards!

...JRF...

[Steven E. Protter]

Shalom,

This plan absolutely can work.

Its just a matter of using lvextend -m 0 and then -m 1 to reduce the mirrors and create new mirrors on the new disk array.

Note: EVA type arrays come with management tools that can do the job more quickly than mirror/ux which is the tool you are using with lvextend.

SEP

[No idea what I'm doing]

James. Your script did the trick. Thanks for everyone's help!

[Emil Velez]

Why did you not just add the new XP luns to the volume group and then just

pvmove /dev/dsk/olddisk
vgreduce /dev/vgXXX /dev/dsk/olddisk

[Mel Burslan]

As an answer to Emil's suggestion: pvmove is a glacially slow command and very prone to corrupting data, should something happen to the process performing the pvmove.

My choice of utilities would be vxdump and vxrestore which are much faster and they do not explicitly destroy the source data as pvmove does. But looks like Mirror-UX worked well for the person who asked the question.

[Torsten.]

IMHO the problem was that the LVOLs are using Allocation PVG-strict/distributed, this means each LVOL is on each disk. For lvreduce the mirror you must specify all involved PVs (all of the PVG).

[No idea what I'm doing]

I am still having much difficulty reducing out these disks from the logical volume. Here is my problem:

Jim, I had been using your script to attempt this. I am essentially trying to run through each logical volume(lvol1-lvol11) and reduce out each of the c4t0d devices. After the first device is reduced(/dev/dsk/c4t0d0), the mirror is broken and the /dev/dsk/c12t0d1 disk device slides into the spot where /dev/dsk/c4t0d0 was. So it goes from this:

--- Logical volumes ---
LV Name /dev/vg20/lvol1
VG Name /dev/vg20
LV Permission read/write
LV Status available/syncd
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 7000
Current LE 875
Allocated PE 1750
Stripes 0
Stripe Size (Kbytes) 0
Bad block NONE
Allocation PVG-strict/distributed
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c4t0d1 175 175
/dev/dsk/c4t0d2 175 175
/dev/dsk/c4t0d3 175 175
/dev/dsk/c4t0d7 175 175
/dev/dsk/c12t0d1 175 175
/dev/dsk/c12t0d3 175 175
/dev/dsk/c12t0d5 175 175
/dev/dsk/c12t0d2 175 175
/dev/dsk/c12t0d4 175 175
/dev/dsk/c4t0d0 175 175

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c4t0d0 00000 current /dev/dsk/c12t0d1 00000 current
00001 /dev/dsk/c4t0d1 00000 current /dev/dsk/c12t0d3 00000 current
00002 /dev/dsk/c4t0d2 00000 current /dev/dsk/c12t0d5 00000 current
00003 /dev/dsk/c4t0d3 00000 current /dev/dsk/c12t0d2 00000 current
00004 /dev/dsk/c4t0d7 00000 current /dev/dsk/c12t0d4 00000 current
00005 /dev/dsk/c4t0d0 00001 current /dev/dsk/c12t0d1 00001 current
00006 /dev/dsk/c4t0d1 00001 current /dev/dsk/c12t0d3 00001 current
00007 /dev/dsk/c4t0d2 00001 current /dev/dsk/c12t0d5 00001 current
00008 /dev/dsk/c4t0d3 00001 current /dev/dsk/c12t0d2 00001 current
00009 /dev/dsk/c4t0d7 00001 current /dev/dsk/c12t0d4 00001 current
00010 /dev/dsk/c4t0d0 00002 current /dev/dsk/c12t0d1 00002 current
00011 /dev/dsk/c4t0d1 00002 current /dev/dsk/c12t0d3 00002 current
00012 /dev/dsk/c4t0d2 00002 current /dev/dsk/c12t0d5 00002 current
00013 /dev/dsk/c4t0d3 00002 current /dev/dsk/c12t0d2 00002 current
00014 /dev/dsk/c4t0d7 00002 current /dev/dsk/c12t0d4 00002 current


to this:

root@rachp23:/root# lvdisplay -v /dev/vg20/lvol1 | more
--- Logical volumes ---
LV Name /dev/vg20/lvol1
VG Name /dev/vg20
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 7000
Current LE 875
Allocated PE 875
Stripes 0
Stripe Size (Kbytes) 0
Bad block NONE
Allocation PVG-strict/distributed
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c4t0d1 175 175
/dev/dsk/c4t0d2 175 175
/dev/dsk/c4t0d3 175 175
/dev/dsk/c4t0d7 175 175
/dev/dsk/c12t0d1 175 175

--- Logical extents ---
LE PV1 PE1 Status 1
00000 /dev/dsk/c12t0d1 00000 current
00001 /dev/dsk/c4t0d1 00000 current
00002 /dev/dsk/c4t0d2 00000 current
00003 /dev/dsk/c4t0d3 00000 current
00004 /dev/dsk/c4t0d7 00000 current
00005 /dev/dsk/c12t0d1 00001 current
00006 /dev/dsk/c4t0d1 00001 current
00007 /dev/dsk/c4t0d2 00001 current
00008 /dev/dsk/c4t0d3 00001 current
00009 /dev/dsk/c4t0d7 00001 current
00010 /dev/dsk/c12t0d1 00002 current
00011 /dev/dsk/c4t0d1 00002 current
00012 /dev/dsk/c4t0d2 00002 current
00013 /dev/dsk/c4t0d3 00002 current
00014 /dev/dsk/c4t0d7 00002 current
00015 /dev/dsk/c12t0d1 00003 current
00016 /dev/dsk/c4t0d1 00003 current
00017 /dev/dsk/c4t0d2 00003 current
00018 /dev/dsk/c4t0d3 00003 current
00019 /dev/dsk/c4t0d7 00003 current
00020 /dev/dsk/c12t0d1 00004 current
00021 /dev/dsk/c4t0d1 00004 current
00022 /dev/dsk/c4t0d2 00004 current
00023 /dev/dsk/c4t0d3 00004 current
00024 /dev/dsk/c4t0d7 00004 current



At that point, I'm in a bad spot because I now have extents on both the old array and the new array. Not extents on just one or other other. I end up having to pvmove /dev/dsk/c12t0d1(new array) back to /c4t0d0(old array). Then I have to remirror them to get the data back on the new array.

So I'm still stuck on how to break the mirror properly so that I can reduce out the old array disks. This would be ten times easier if lvreduce would take a physical volume group as an option.

[TTr]

You are about to use pvmove to move the "c12" lun back to the "c4" lun. Why not use pvmove to move the "c4" luns forward to the other "c12' luns? No mirroring is needed.

[Patrick Wallek]

Would the following work:

# lvreduce -m 0 /dev/vg20/lvol1 /dev/dsk/c4t0d0 /dev/dsk/c4t0d1 /dev/dsk/c4t0d2 /dev/dsk/c4t0d3 /dev/dsk/c4t0d7

where you list ALL disks you want to remove from the mirror on the command line.

[No idea what I'm doing]

Yes! Thank you, Patrick. That was what I needed. I wasn't putting all of the PVs in the one lvreduce command. I didn't pay enough attention to the ellipses after PhysicalVolumePath in the lvreduce man page.

This is the script I'm now using to handle this.

# FILE should contain all the primary links to the disks in the VG
# FILE2 should contain both the primary and alternate links
typeset VG=vg20
# FILE should contain only the primary links
FILE=/tmp/newdisks
# FILE2 should contain both the primary and alternate links
FILE2=/tmp/newdisks2
for LV in 3 4 5 6 7 8 9 10 11
do
echo "Breaking mirrors, lvreducing PVs..."
lvreduce -m 0 /dev/${VG}/lvol${LV} $(cat ${FILE})
done
echo "VGreducing the PVs from $VG..."
vgreduce $VG $(cat ${FILE2})