Changing array slot of disk in a mirror set?

[mvpel]

Hi everyone,

I've got a system which has a set of volumes mirrored across two external disk arrays, and I want to change it so that all the disks are in a single array, so that I can use the second array for another system. I have enough empty slots in the first array to accommodate all the disks.

I assume that it's not quite as simple as just unplugging a PV in the mirror from the second array and moving it to a free slot in the first array - or is it?

Are there any manual steps (import? rescan?) needed to get the LVM to recognize the new device path to the existing physical volume, or is it smart enough to recognize the PV label on the disk at its new /dev/dsk path and bring it back into the mirror without any drama?

I'd prefer to do it without rebooting, needless to say.

Thanks for any insights you can offer!

[likid0 · Accepted solution]

If i understood correctly you have a set of lvols mirrored using mirrordisk/ux with one pv on one array and another pv in the other array ??

if this is the case you just have to remove the mirror and vgreduce the disks from the array you want to leave empty:

remove the mirror from all the lvols that have disks beloging to the old array
lvreduce -m 0 /dev/vglol/lvol /dev/dsk/diskoldarray
once no lvol is using the disk reduce the vg:
vgreduce /dev/vglol /dev/dsk/diskoldarray


And if you wan't to redo your mirror with a disk in the new array then, present the new lun,ioscan, insf ..
vgextend /dev/vglol /dev/dsk/newdiskarray
lvextend -m 1 /dev/vglol/lvol /dev/dsk/newdiskarray

no need to stop.

[TTr]

> ...mirrored across two external disk arrays,

It looks like you are using LVM mirroring with external disk enclosures (not arrays).
You need to provide more details.
If they are mirrored using LVM , then the relocation of the disks needs to be handled within LVM. The vgexport/vgimport is one method to use. If you have spare disks, vgextend/pvmove/vgreduce is also an option.

[Pete Randall]

How about pvmove?

"move allocated physical extents from one LVM physical volume to other physical volumes"

You would need to set up the new logical volumes (in the same VG, of course), but then you should be able to pvmove the second array's LVs to the new LVs. Then you do what you like with the old LVs.



Pete

[mvpel]

Yes, the volumes are mirrored with MirrorDisk/UX.

My hope is that I'd be able to simply unplug the mirror disk from one array, plug it into the other array, perform the appropriate incantation, and it would recognize and resync the existing mirror onto the relocated volume - without having to break the mirror, move the disks, and then recreate it.

Hope that's a bit more clear.

[mvpel]

I also considered the pvmove option with new disks, but that's suboptimal because it would involve additional paperwork in changing the serial numbers recorded in the hardware baseline for the system, but it's certainly a possibility.

[Pete Randall]

> My hope is that I'd be able to simply unplug the mirror disk from one array, plug it into the other array, perform the appropriate incantation, and it would recognize and resync the existing mirror onto the relocated volume

Sorry to disappoint but that's not going to happen.


Pete

[mvpel]

Thanks very much for the info!

[Pete Randall]

The more I think about it, the more I have to wonder if a vgexport/vgimport might not be able to find the switched disks:

vgexport -s -m /tmp/vgNN/map
If necessary, recreate /dev/vgNN:
mkdir /dev/vgNN
mknod /dev/vgXX/group c 64 0xNN0000
Then reimport the VG:
vgimport -s -m /tmp/vgNN/map /dev/vgNN

The -s option should go out and look at all the disks LVM headers and "discover" the newly switched disks.

Possibly worth a try but I would want to have a good backup - which you ought to have in any case!


Pete

[mvpel]

Hm, interesting possibility...

As luck would have it I now have another system that's installed but not yet on the network that I can use to see what happens when a PV changes hardware paths. I'll try it out and post the results.

[Emil Velez]

When you say disk arrays that makes me wonder why you need mirroring. If you just moved disks from one FC or scsi card to another and they were just regular disks tha the OS sees you would use a procedure like what is below.

if a physical volume changes paths LVM will not recongize it automatically

1. umount all filesystems
2. deactivate volume group
3. export volume group with the -s option

vgexport -s -m /tmp/vgxx.map vgxx

4. move the disks
and plug them in
5. ioscan
6. insf -e
to create the new device files at the new location

Now I would suggest that you spend a little time making sure with ioscan -funCdisk that the disks are all seen at the new location.

7. mkdir /dev/vgxx
mknod /dev/vgxx/group c 64 0xXXXXXXX
8. here is where the magic happens
vgimport -s -m /tmp/vgxx vgxx

the -s option causes the system to scan all of the disks for the VGID. The disks already have a PVID which will tell lvm what pieces are on what disks but the VGID tells LVM what disks belong in the volume group.

[Vishu]

Hi,

well, simple and straight and reliable.

1. Just do lvreduce and then vgreduce.
2. Your disks are now free to move to anywhere.
3. Connect the disks to the new array.
4. Do ioscan and 'insf -eC disk'
5. Recognize the new disks and do pvcreate on those.
6. Extend those PVs to the VGs and do mirroring with lvextend.

Thanks

[AVV]

Hi,

Please try with the vgimport/export options with -s switch. It would be easier than lvreduce/vgreduce.

[mvpel]

"When you say disk arrays that makes me wonder why you need mirroring. "

Maybe I'm not using proper HP terminology - perhaps "enclosure?" It's a Storageworks 2405 JBOD enclosure, no hardware mirroring or RAID.

Thanks for the suggestion, I think that what you wrote, Emil, is what I was envisoning. I'll let everyone know what turns up from my test.

[mvpel]

My notion of being able to move the disks while the system is online was rather unceremoniously quashed by the realities of 11.11 LVM - the system recognized the new disk in the new path, but it was still looking for the disk in its old path, naturally, waiting for it to be replaced, and was having a hard time recognizing the disk in its new location as a proper part of the volume group.

Once I started to delve into the process of assuaging the system's confusion, it quickly became apparent that doing paperwork on new disks, doing paperwork on the old disks, and doing a pvmove would be much, much easier.

Thanks for your input, everyone, I appreciate it!

[mvpel]

The alternative of breaking the mirror and recreating it is also appealing, but given the track record of some of these disks I'd be concerned about being unmirrored while waiting for the resync to complete.