Break Root Mirror and Re-Create It

[Steven Chen_1]

Hi,

I created mirror on root volumn disk with the 2nd disk. Before the server is on production, I would like to know if I can break the mirror, boot from the 2nd disk, then re-mirror the 1st root disk to the 2nd one.

Is it possible and how to impletement it? It seems to be a fire drill, but somebody on top want to see it work.

Should help comes in, very appreciated.

Steven

[Paula J Frazer-Campbell · Accepted solution]

Steven

First of all is the mirror disk bootable?

lvlnboot -v To check if you see both the disks as boot disks.

Paula

[A. Clay Stephenson]

You can certainly do this by doing an lvreduce of each LVOL (specifying the desired disk) and then a vgreduce BUT that really is not necessary. A much better test is to physically remove the primary disk and then boot the machine. If you have done all the steps correctly (including adding -lq to override quorum to the mkboot -a command then your system should be able to boot using the alternate device.

By the way, I suggest that you add the -lq to the mkboot -a string of both the primary and alternate boot disks.

[S.K. Chan]

Before you break the mirror PLEASE verify (with both disk mirrored) that you can boot up from the 2ndary mirrored disk as well as primary mirrored disk. Just change the boot path accordingly. Basically we just want to make sure bot disks boots up fine.
Now the steps...
o c1t2d0 is primary mirrored disk & c2t2d0 is the 2ndary mirrored disk.
o Assuming (lvol1=/stand;lvol2=primary swap;lvol3=/;lvol4->lvol8 representing the rest of root's FS)

To break the mirror..
# lvreduce -m 0 /dev/vg00/lvol2 /dev/dsk/c2t2d0
# lvreduce -m 0 /dev/vg00/lvol1 /dev/dsk/c2t2d0
# lvreduce -m 0 /dev/vg00/lvol3 /dev/dsk/c2t2d0
==> Repeat for the rest of the lvols.
# vgreduce vg00 /dev/dsk/c2t2d0
# lvlnboot -v
==> Just checking. c2t2d0 should be out.

Change boot path ..
# setboot
=> Display boot path.
# setboot -p 0/0/2/0.2.0
=> Change primary boot path to c2t2d0
# setboot
=> Check again.

Now shutdown -r the server.

After it comes up .. re-do the mirroring process ..
# pvcreate -f -B /dev/rdsk/c1t2d0
# mkboot -l /dev/rdsk/c1t2d0
=> Create boot track (place boot utilities in boot area), -l means treat device as LVM disk.
# mkboot -a "hpux -lq" /dev/rdsk/c1t2d0
=> Add AUTO file in boot LIF area, "lq" option is to bypass quorum check.
=> Repeat for primary disk (c2t2d0) as well.
# lifcp /dev/rdsk/c1t2d0:AUTO -
=> Check content of AUTO file. Repeat for primary disk (c1t2d0).
# vgextend /dev/vg00 /dev/dsk/c1t2d0
=> Adding disk to root VG.
# pvdisplay -v /dev/dsk/c1t2d0
=> Note of the order of LVs (assuming it's lvol1,lvol2,lvol3,lvol4->lvol8)
# lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c1t2d0
=> Repeat for lvol2->lvol8
# lvlnboot -r /dev/vg00/lvol4 /dev/vg00
# lvlnboot -b /dev/vg00/lvol1 /dev/vg00
# lvlnboot -s /dev/vg00/lvol2 /dev/vg00
# lvlnboot -d /dev/vg00/lvol2 /dev/vg00
=> Update BDRA (root, boot, swap and dump)
# lvlnboot -v
=> Check to make sure both disks are listed for root, boot and swap definition.
# setboot
=> Display boot path an set the boot path to the desired path with "setboot -a|-p"

[Steven Chen_1]

Thanks all for the overwhelming help! First disk is the original boot disk, 2nd disk can be booted too.

[S.K. Chan]

I did not really read you question well. It sounds like you want to have first hand try out in case of disaster ? Well the real test is (in the event of a failed mirrored root disk) usually NOT haveing to break a mirrored disk but rather how to replace it. When either one of your root disk failed, error message will start to pop up in dmesg, syslog and you would see stale extents. Depending on whether you can hot swap a repalcement disk or not, you would have to go through a few steps before everything is back in operational. After replacement of the "bad" disk ..
# pvcreate -f -B /dev/rdsk/
# mkboot -l /dev/rdsk/
# mkboot -a "hpux -lq" /dev/rdsk/
# vgcfgrestore -n vg00 /dev/rdsk/
=> recovers LVM info to disk
# vgchange -a y vg00
=> enables new root disk
# vgsync vg00
=> configure LVs and copies data from primary disk, this will take awhile
Both disks are now in sync.

[Steven Chen_1]

So there are two options to test it out:

1) remove one drive and put it back on;
2) break the mirror and re-mirror the root drive

The question is: would recreating the first drive really damage it? Is it worth the test?

[A. Clay Stephenson]

Actually, an even better question is "How do I know that I did everything correctly when I re-mirrored?"
You still have to test by booting from each disk so simply trying to boot from both disks (and doing a vgsync) after replacing the drive accomplishes everything and leaves you in a known good state.

[S.K. Chan]

No recreating the 1st drive would not damage it but in my opinion it's not worth the trouble (ie breaking the mirror and then recreating it). This is what I usually do to simulate my mirrored root drive failure.
- Drive A (primary); Drive B (secondary)
- Primary boot path = A ; 2ndary boot path = B

a) Test both A & B boot up by interrupt the boot process and select the disk to boot up.
b) Unplug or disconnect (one disk at a time) and make sure system continues to run. In my case it's easy for me to test this because the root drives are accessible from my front cage panel. Test is done on A then on B.
c) Unplug or disconnect (one disk at a time) and if the system continues to run fine (ie if test b above passes), reboot the system. With escape quorum set, the server should be able to boot up fine even with one disk disconnected. When system is online plug the disk back (need to "vgchange -a y vg00" to include it back).

If a,b and c all passes I think you're in good shape. And now you can test the actual replacement of a failed root disk with the process that I mentioned earlier.