How can we remove PV's from a VG that has been mirrored?

[MTS Infrastructure]

Here's the problem, we are currently in the process of migrating our storage from one EMC san array to another, but the new storage might have a higher LUN id, than the current storage. We have setup persistent naming using the /etc/multipath.conf file. The process we are using for the migragtion is the following:

1. add new EMC drives from the new array
2. scan for the new drives
3. erase any data on the new drives
4. format the new drives the same as the old (sfdisk -d /dev/sd|sfdisk --force -q /dev/sd
5. copy the boot partition to the new drives (dd if=/dev/sd1 of=/dev/sd1 bs=4096)
6. create PV's to add to the VG's (kpartx -a /dev/mapper/hdisk; pvcreate /dev/mapper/hdisk
7. add the new PV's to the VG's (vgextend /dev/mapper/hdisk)
8. mirror the LV's (lvconvert --mirrorlog core -m1 /lv)
9. add the new boot partition to the /boot/grub/device.map, and run grub-install to install the boot block
10. Edit the /etc/fstab file to point to the new partitions
11. run mkinitrd to create a new initrd file
12. reboot the server, and point to then new drives in the ILO

This procedure works fine, and the server will come up on the new drives for the rootvg. At this point, we can unmirror (lvconvert -m0 /lv), and vgreduce the VG, and then remove the old SAN drives.

The problem occurs with trying to remove the old drives from our uservg. During the boot process, it will scan for VG's on all the PV's, and the first one it finds becomes the primary PV for the VG. So, this means that if the old drives have a lower LUN id, they will be the primary PV. When you unmirror, you can only remove the new drives?

We have tried many different things:

1. breaking the mirror by removing the drives, the server can panic if the drives are being used.
2. vgexport, shutdown, remove old drives, reboot, vgscan --partial, vgimport
3. vgcfgbackup --file, edit the file and remove the old PV's, vgcfgrestore --file
4. change the filter in /etc/lvm/lvm.conf

Any ideas on how to remove the drives from the VG, or have the server only scan for the new PV's on boot?

Thanks.

[Steven E. Protter]

Shalom,

You need to pull the LUN's back that presented the disk drives you are trying to remove from the system.

So long as EMC presents the LUNS, the system will see disk.

EMC does have utilities that can be run to stop presentation of the disks to the system once you are done with them. Utilities or GUI interfaces to make things right.

SEP

[MTS Infrastructure]

Thanks for your response, but we do remove LUN's, but it causes issues.

(1) If we remove the LUN's after the mirroring, the system can panic

(2) If we break the mirror (lvconvert -m0), shutdown, and remove the LUN's, when we boot, it can not find the VG's. We have to boot into rescue mode, scan for the VG's, mount...etc.

What we are looking for is a way to remove the PV's from the VG's (both rootvg and uservg), and then remove the LUN's while the server is online, or after a reboot. We can accomplish for the rootvg, but not the uservg, if the LUN id's are higher.

Thanks.