Online LVM disk mirrior replacement, what about the 2nd node

[hydrocct]

I replaced a bad disk in a mounted exlcusive mirrored VG on the primary node of a 2 node cluster, in a FC10 (yes we're still using that) cabinet. On the primary node, I went through the fcmsutil replace_dsk procedure. Because the disk was not new, I had to force pvcreate on the PV. I also lvreduced all lvs that used the bad disk and then lvextended -m 1 and vgextended everything and now all is well, i.e. the VG is reynced. I also had to vgcfgrestore and also do a vgscan to rebuild the lvmtab. All online while the VG was mounted. Nothing exceptionnal, but still demanding when none of the commands in the written procedure work (probably cause the replacement disk was not new).

Anyhow, my question is this: on the secondary node (on which the VG is currently not mounted), apart from having to do
fcmsutil /dev/td0 replace_dsk -l 7 to stop it from complaining about changed WWN,
what else should be done? Should the lvmtab be also rebuilt? Any files to export?
I just want to make sure everything goes well when the package eventually bounces to that secondary node.

[Ramesh S · Accepted solution]

Hi

On primary node:

>Please export that VG in preview mode to create map file.

vgexport -v -p -s -m

[psreedhar]

As you have just replaced the disk in the same slot of old disk, you don't have to do any thing. Simply running fcmsutil /dev/td0 replace_disk is enough.
Once the disk is shown as claimed in ioscan, run vgchange -a y , on the VG which has this disk. That will solve your probem.

[Sajjad Sahir]

do u have mirroring
plese reply soon

[Siju Vadakkan]

Hi hydrocct,

You don't need to do anything at LVM level on the adoptive node since u replaced the new disk with the same slot and you can issue the fcmsutil command.

Hi Sridhar,

You can't activate a volume group in read/write(vgchange -a y Vgname) mode which is already mounted in exclusive mode on one of the cluster node.What you can do is , just activate the Volume group in read only mode (vgchange -a r vgname) on the adoptive node to verify the configuration.

[Torsten.]

I doubt about the pvcreate command - not needed!

The only thing you need to do on the other nodes is the "fcmsutil ... replace_dsk ..." and ensure you can see this disk CLAIMED in ioscan.

[Sajjad Sahir]

removed

[Stephen Doud]

Assuming the device special file to the replaced disk did not change, you should not have to do anything more than correct the FC-related messages on the 2nd node.
The reason being that /etc/lvmtab only contains a list of the volume groups, device files of disks that belong to them and the VGID of each VG (the last item cannot be seen by the 'strings' command.
The fact that you reduced mirrors off of the problem disk doesn't matter, since mirror information is not kept in a file on a file system, but in the LVM metadata on the disks in the volume group.

[hydrocct]

I did what Ramesh suggested which, as may of you indicate, is probably not needed, but shouln't do any harm Anyhow, I 'll have to wait for some down time or an application failure to test it.

[Siju Vadakkan]

You don't need downtime to verify the configuration, Just activate the VG in read only mode on adoptive and compare with primary node,after that u can deactivate the Vg on the adoptive node.

#vgchange -a r vgname
#vgdisplay -v vgname
#vgchange -a n vgname

[hydrocct]

The -r option does not exist on my systems (HP-UX 11.0)