Replacing Failed mirrored root disk - online

[Jayasuntar]

Hi..

We want to replace the failed mirrored root disk. I got lot of answers in ITRC, all are talking abt rebooting the system in lq mode and restore vginfo. and do the vgsync.

We can not reboot or shutdown the system, is there anyway we can replace mirror root disk online?..

thanks in advance.

Regards

[Stf]

look at this attached document...

Stf ;-)

[Eknath]

HI Jayasuntar,

pls find the steps for replacing the mirrored disk online

Cheers !!!
eknath

[Ralph Grothe]

Thanks Stf for supplying your mode d'emploi for root disk replacement because I'm always keen to improve my foreign language skills.
I haven't known unitl know that in Bahrain they speak French, but this mitigated my ignorance:

http://www.ambafrance-bh.org/article.php3?id_article=234

[Jayasuntar]

Hi Stf,

Thanks for your document.

But, u can not do vgcfgrestore when the vg is active in another way, vg should be deactivated before doing vgcfgrestore.

I dont want to do that. W/O deactivating vg and rebooting the system I need to restore the mirror back.

Regards

[Pete Randall · Accepted solution]

So, you're saying that this vg is activated in shared mode? Why would you be sharing a root VG?

You HAVE to do a vgcfgrestore in order to restore the VG structures onto your newly replaced drive so that you can then resync the mirrors.


Pete

[Jayasuntar]

Dear Pete,

No it is not activated in shared mode.

man vgcfgrestore says to do vgcfgrestore, vg to be deactivated.

Regards

[Patrick Wallek]

A VG does NOT have to be deactivated to do a vgcfgrestore. I have done these numerous times with a vg active.

[Pete Randall]

Actually the man page says the VG can't be activated in shared mode. It does NOT say that it has to be deactivated. Just run vgcfgrestore.


Pete

[A. Clay Stephenson]

You can certainly run vgcfgrestore while the VG is activated. I have not shutdown or deactivated a VG to replace a disk (including many boot disks) in well over 6 years.

[Sanjay_6]

Hi,

You can replace the root mirror disk online without reboot if the disk is hot-swappable. If not then you have to reboot the server to unplug the disk.

Try this link from itrc, it mentions different situation for a lvm disk replacement.

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000073905420

The itrc doc id is KBAN00000347.

Hope this helps.

Regds

[vinod_25]

hi jai

The LVM Online Disk Replacement (OLR) feature provides new
methods for replacing or isolating LVM disks within an active
volume group. pvchange(1M) will have new option, -a, to
detach/attach specified PVs (Physical Volumes) on an active
volume group.

The following is an example procedure to replace a faulting/
unstable physical disk on an LVM-mirror:

# pvchange -a n

or

# pvchange -a N (to detach all paths
to the disk)

This makes the physical disk unavailable for this volume group.
LVM stops sending any I/Os to this disk. The detached disk can
be safely tested, replaced, and re-initialized. After the disk
is replaced, initialize the partition and boot area, (if it is
part of a bootable VG), restore the LVM configuration information
with the vgcfgrestore(1M) command, and attach the physical volume
with:

# pvchange -a y

# vgchange -a y

When LVM recognizes the new disk, it will start to synchronize
the mirror data.

The LVM OLR feature is enabled with the following patches:

HP-UX 11.11:

[PHKL_31216/PACHRDME/English]
s700_800 11.11 LVM Cumulative Patch; LVM OLR; SLVM 16 Node

[PHCO_30698/PACHRDME/English]
s700_800 11.11 LVM commands cumulative patch; LVM OLR

HP-UX 11.23:

[PHKL_32095/PACHRDME/English]
s700_800 11.23 LVM Cumulative Patch; OLdR enablement

[PHCO_31709/PACHRDME/English]
s700_800 11.23 LVM commands patch

Regards

Vinod K

[Geoff Wild]

Have you solved your issue? As others have mentioned - you can replace online - if disk is hot swap.

Here's some options I use:



After failed disk has been replaced by HP (assuming the disk is c1t5d0):

vgcfgrestore -n /dev/vg00 /dev/rdsk/c1t5d0

mkboot /dev/rdsk/c1t5d0

mkboot -a "hpux -lq (;0) /stand/vmunix" /dev/rdsk/c1t5d0

vgchange -a y /dev/vg00

lvlnboot -Rv /dev/vg00

vgsync /dev/vg00


Now,

1) If the disk is completely "dead", such as if you run ioscan and status is "no_hw" then you can hot swap the disk online.

2) However in circumstances where the disk has not fully failed please do one of the following to avoid data corruption :

a) reduce mirror before replacing the disk
b) deactivate VG before replacing the disk
c) shutdown system to replace the disk



If you have to rebuild the mirror:

Say /dev/dsk/c0t6d0 is the root disk and /dev/dsk/c3t6d0 is the disk you want to make a mirror of.

1. Create a bootable LVM disk to be used for the mirror.
pvcreate -B /dev/rdsk/c3t6d0
2. Add this disk to the current root volume group.
vgextend /dev/vg00 /dev/dsk/c3t6d0
3. Make the new disk a boot disk.
mkboot -l /dev/rdsk/c3t6d0
4. Copy the correct AUTO file into the new LIF area.
mkboot -a "hpux -lq (;0)/vmunix" /dev/rdsk/c3t6d0
5. Mirror the boot, root and primary swap logical volumes to the new
bootable disk. Ensure that all devices in vg00, such as /usr, /swap,
etc., are mirrored.
The following is an example of mirroring the boot logical volume:
lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c3t6d0
The following is an example of mirroring the primary swap logical
volume:
lvextend -m 1 /dev/vg00/lvol2 /dev/dsk/c3t6d0
The following is an example of mirroring the root logical volume:
lvextend -m 1 /dev/vg00/lvol3 /dev/dsk/c3t6d0
6. Update the boot information contained in the BDRA for the mirror
copies of boot, primary swap and root.
/usr/sbin/lvlnboot -b /dev/vg00/lvol1
/usr/sbin/lvlnboot -s /dev/vg00/lvol2
/usr/sbin/lvlnboot -r /dev/vg00/lvol3
7. Check if the BDRA is correct.
/usr/sbin/lvlnboot -R /dev/vg00
8. Verify that the mirrors were properly created.
lvlnboot -v /dev/vg00
The output of this command is shown in a display like the following:
Boot Definitions for Volume Group /dev/vg00:
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c0t6d0 (1/0/0/3/0.6.0) -- Boot Disk
/dev/dsk/c3t6d0 (1/0/1/0/0/1/1.6.0) -- Boot Disk
Boot: lvol1 on: /dev/dsk/c0t6d0
/dev/dsk/c3t6d0
Root: lvol3 on: /dev/dsk/c0t6d0
/dev/dsk/c3t6d0
Swap: lvol2 on: /dev/dsk/c0t6d0
/dev/dsk/c3t6d0
Dump: lvol2 on: /dev/dsk/c0t6d0, 0

vgdisplay -v vg00

Then lvextend for all other lvols:

lvextend -m 1 /dev/vg00/lvol4 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol5 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol6 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol7 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol8 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol9 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol10 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol11 /dev/dsk/c3t6d0


Rgds...Geoff


BTW - welcome to the forums. Please assign points to answers. Points assignment helps others determine how good an answer is.

http://forums1.itrc.hp.com/service/forums/helptips.do?#28

[Senthil Kumar .A_1]

Hi Jayasuntar,

If you have a hot swappable or a free disk on the system is added advantage...

you can run vgdisplay -v and check for the drive that has gone bad..

then remove the mirror as follows..ex:

lvreduce -m 0 -k /dev/vg00/lvol1 /dev/dsk/c2t3d0 2 &

NOTE:
-k is important, used in special cases if the pv is unreachable see (man lvreduce).

& is important, because u need to run it in background, so that u can kill the llvreduce process if it hangs after it displays that mirror was removed.

2 : used at the end represents the pv key.. in your lvdisplay -v a LV, u will find 2 to 3 disks mentioned..the disk in the first column represents "1" ,the one in second column represent "2".. so in this case i have assumed ur second PV to have problems..

For more details...visit..

http://docs.hp.com/en/A5191-96018/ch07s02.html

[Jayasuntar]

Thanks to all for your inputs.

We replaced the disk online.

thanks a lot once again.