Mirror Disks for Disaster Recovery when Patching

[phil atkinson]

I have a RP5400 server HPUX11 with software mirroring.
I want to patch the server. The patches are fragile and I want to be able to move quickly back to a working system in case of a problem.
I plan to remove one of the mirror disks while performing the patch.
If the patch is successful I will replace the disk and sync the mirror. If not I want to boot off the backup disk and sync the patched disk with the backup (go back to starting point).
In the latter disaster scenario is it enough to boot off the backup disk and run vgsync vg00?

[Victor Fridyev]

Hi,

If your server is not mission critical, you can do the following:
reduce mirroring, copy system disk to the second one by dd, upgrade the system and, if everything is OK, restore mirroring.
I use dd copy instead of mirroring. Firstly, I always have yesterday disk copy, secondly I don't have to pay for mirroring license.
HTH

[Bob Ingersoll]

Hi,

Your approach using mirroring should work fine.

[phil atkinson]

I tested the procedure today and simulated the failure condition.

The vgsync failed with the following

vgsync /dev/vg00
Resynchronized logical volume "/dev/vg00/lvol1".
Resynchronized logical volume "/dev/vg00/lvol2".
Resynchronized logical volume "/dev/vg00/lvol3".
vgsync: Couldn't re-synchronize stale partitions of the logical volume:
Device offline/Powerfailed

vgsync: Couldn't resynchronize logical volume "/dev/vg00/lvol4".
vgsync: Couldn't resynchronize volume group "/dev/vg00".
LVM: vg[0]: pvnum=2 (dev_t=0x1f020000) is POWERFAILED
# LVM: Recovered Path (device 0x1f020000) to PV 2 in VG 0.
LVM: Restored PV 2 to VG 0.

I think the only way out of this is to Ignite the server?

[Kevin Wright]

If you booted the system successfully off of your backup disk, then there isn't any reason to re-install/ignite the OS. You seem to be having an issue with the target mirror disk, potentially a HW problem. Otherwise, reduce it out of vg00/pvcreate -B, and re-mirror.

[John Guster]

This is a practical way to patching a system. Here is the outline:
1. get boot definition: lvlnboot -v vg00
2. /usr/sbin/sync
3. split the logical volumes: lsdplit /dev/vg00/lvol9...do lvol3 last
4.fsck -F hfs /dev/vg00/lvol1b...do for each one
5. mount /dev/vg00/lvol1b /tmp make sure mountable, then unmount them
6.patching
7. patching successful, lvmerge /dev/vg0/lvol9b /dev/vg00/lvol9
8. patching failed do:
a)boot to lvm mode, mount /dev/vg00/lvol3b /tmp
b) vi /tmp/etc/fstab to change lvol1 to lvol1b for all LVs
c)remove boot definition:lvrmboot -r vg00
d)add the split LVs: lvlnboot -b /dev/vg00/lvol1b /dev/vg00...do -r -s -d
e) verify lvlnboot -v vg00
f) change the /tmp/stand/bootconf to point the splpit disk
g)lvlnboot -R vg00
h)reboot the system to boot from the split disk by interrut the boot sequence
i) once system is up, lvmerge /dev/vg00/lvol9 /dev/vg00/lvol9b, do for all LVs
j)lvlnboot -v vg00 to verify again
k)reboot to lvm mode, to change LVs name back to normal: cd /dev/vg00; mv lvol1b lvol1;do for all Lvs
l) update /etc/fstab to go back to original
m)shutdown -ry 0 should bring your system back to the stage before patching.

The other way is much straigh forward is make an ignite image. Back out will be boot from the ignite tape.
Hope helps.
Cheers.

[A. Clay Stephenson · Accepted solution]

A far better way is to leave you lvol's mirrored and do a raw dd copy of your boot disk to a 3rd (lifeboat) disk. Although this is a dirty copy, I've never had one fail to boot after the system does the automatic fsck's. If anything goes wrong, you simply pull the boot (and mirror) disks and put the lifeboat in the boot slot and the system is rebooted. You are back in action in a fraction of the time that an Ignite takes. I make a raw dd copy every weekend on all my boxes --- and before any patches are applied. I don't like to leave the machine vulnerable by defeating the mirroring; after all, most patches work just fine.

[phil atkinson]

Thanks for your advice.
I think I will abandon the disk mirroring backup strategy. The lifeboat looks a better method. Have to persuade the boss to buy 3x36GB disks though.