Changing the pv's in a vg

[Terrence]

I have a volume group in a service guard cluster.

I need to remove the 6 pv's (luns) in the vg and replace them with 6 other pv's.

To make this fun I can't bring the cluster or package (application) down while doing this.

There are 3 lv's in the vg.

I can move the data in the lv's, and then temporarily, delete the lv's. Once the 3 lv's are gone, I want to add the 6 new pv's and then remove the 6 old pv's. Once I've got that done I want to recreate the lv's with the same names that service guard expects.

Of course I'll vgexport/vgimport to resynchronize the nodes.

However I seem to recall that you can never remove the 1st pv in a vg. True?

Alternatively since service guard is not monitoring the volume group, perhaps I should just move all the data, blow away the vg and then quickly rebuild it with the new pv's. As long as the names are all the same, and the cluster lock disk isn't in that volume group I should be ok.

Any suggestions?

[Sandman!]

Better yet...mirror the LVs onto the new PVs with lvextend(1M) and then lvreduce(1M) the old PVs. This way you don't incure any downtime and the procedure is safe and reliable. In the end vgexport(1M) all the clustered VGs on the primary node in preview mode and vgimport(1M) them onto the each of the other nodes in the cluster.

[James George_1]

Hi

If you have enough down time, dd is also a good option to move your LVs to the new PVs.

here is an example:

# pvcreate < your disk)
# vgextend .....
# lvcreate
# lvextend ...
# newfs -F vxfs /dev/vgXY/rlvolXY #
# umount /XYZ( the existing FS)
# dd if=/dev/vgXX/Old_lv of=/dev/vgXY/new_lv bs=512k &
# mount /dev/vgXXX/newlv /FS
# fsadm â F vxfs â o largefiles /newFS

Remember to do what ever you have to do to make thiese disk cluster aware !!

Rgds / James

[Tom Danzig · Accepted solution]

If you can add the new devices to the volume group (i.e. max physical volumes has not been reached; disk/LUN sizes are the same so you will not run into max PE/PV issues), then you can use pvmove to move the extents off the old devices and onto the new. Once completed, pvremove the old devices out of the VG and do the vgexport/vgimport to the other nodes to keep MC/SG happy.

I've done this many times. Wroks like a champ and provides 100% uptime.

[Tom Danzig]

Correction for typo above.

"pvremove the old devices out of the VG"

should read

"vgreduce the old devices out of the VG"

I can't believe we still cannot edit posts here!

[Terrence]

Tom,

Sounds like the best way.

Two follow up questions.

1.) Man page has this comment:
pvmove cannot be performed if the volume group is activated in shared
mode.

I assume I can just issue a vgchange -c n /dev/vg02. Yes?

2.) Can I remove the first pv used in the vg?

[Sandman!]

Imho...I prefer the lvextend(1M) in case the pvmove(1M) operation is interrupted then you will end up with corrupted source and destination LVOLs and will have to restore from backup. And if the LVOLs are striped then you need to use dd(1) as shown in a previous post.

[Tom Danzig]

1.) Man page has this comment:
pvmove cannot be performed if the volume group is activated in shared
mode.

I assume I can just issue a vgchange -c n /dev/vg02. Yes?

Most likely, your VG is in exclusive mode (use vgdisplay to confirm). If in shared mode, as per the man page, you cannot do this procedure. If the VG is in exclusive mode, the vgchange command you listed is unnecessary.

2.) Can I remove the first pv used in the vg?

Yes. I have done this in the past w/o issue.

[Tom Danzig]

Per HP's document "When Good Disks Go Bad":

"pvmove is not an atomic operation, and moves data extent by extent. If pvmove is abnormally terminated by a system crash or kill -9, the volume group can be left in an inconsistent configuration showing an additional pseudo mirror copy for the extents being moved. You can remove the extra mirror copy using the lvreduce command with the â m option on each of the affected logical volumes; there is no need to specify a disk."

Therefore, your VG/data will not be corrupted.

I don't think stiped LV's matter although I am not 100% sure. Since al you are doing is moving phisical extents from one device to another, IMHO it should not matter.

[Sandman!]

Tom...thanks for clearing the confusion about data corruption with pvmove. So one can recover if the pvmove ends abnormally. However pvmove cannot copy over striped lvols...you need to use dd for that.

[Terrence]

Thanks all,

Tom you were correct I was confusing exclusive with shared mode.

I don't think the lv's are striped, but they do extend onto multiple pv's.

How do I confirm they aren't striped?

[Tom Danzig]

lvdisplay will tell if they are striped. Look for :

Stripes 0

[Sandman!]

# lvdisplay /dev/vg??/lvol?? | grep Stripes

...no striping if the "Stripes" and "Stripe Size" parameters show zero.

[F Verschuren]

hi,

I mis one step, Please check if you cluster lock disk in one of the disks that you replease, if this is the case you have to reaplay the cluster whit new cluster lock disks, This setting wil not go over with mirroring