can't restore mirror after drive failed

[Paul Wentland]

I had to replace a mirrored drive in vg00.

Two issues:
# vgcfgrestore -n vg00 /dev/rdsk/c1t2d0
vgcfgrestore: Mismatch between the backup file and the running kernel:
Kernel indicates 4 disks for "/dev/vg00"; /etc/lvmconf/vg00.conf indicates 2 disks.
Cannot proceed with the restoration. Deactivate the Volume Group and try again.

How to get rid off two hidden drives in kernel?

# vgsync vg00
vgsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
vgsync: Couldn't resynchronize logical volume "/dev/vg00/lvol1".
vgsync: Couldn't resynchronize volume group "vg00".

vgdisplay shows LV Status - available/stale.

I am stuck, need help please.

[sujit kumar singh]

HI
Please post the O/p as of the following commands
#ioscan -fnCdisk
#strings /etc/lvmtab
#vgdisplay -v vg00
regards
sujit

[Mark McDonald_2]

Paul

I've seen a similar problem. I ended up force ably removing the mirror config and re-mirroring from scratch to the new disk.

I remember the kernel out of sync with running kernel error being a pain. And the only way I found to get rid of it was to reboot the box. Others may have a better Idea.

Good Luck
Mark

[vijay alur]

Hi Paul,
can you write what all steps you followed for the faulty mirrored root disk replacement and the steps you have taken for root disk mirroring. May b u missed on some steps....

It might have happened that your /etc/lvmtab file is not updated with the current configuration. To update the /etc/lvmtab to the current configuration you can recreate the /etc/lvmtab with the help of vgscan command, but before doing that please save a backup copy of the original /etc/lvmtab and then run the vgscan -v command.

if you have followed the proper steps for the root disk mirroring then this should resolve the issue by just recreating the /etc/lvmtab file.

if you want here are the steps for root disk mirroring for PA-RISC servers, but u need to have mirrordisk-ux installed for that.

Following are the steps for the Root Disk mirroring for PA-RISC Servers.

1. Add a disk to the server.
2. Check with ioscan -fnC disk if the disk is detected and its claimed state.suppose the disk is detected with the device file name c2t15d0.
3. Now give the command pvcreate -B /dev/rdsk/c2t15d0 to make that disk as LVM bootable disk.
4. Now include the PV /dev/dsk/c2t15d0 in the VG00 which is the root VG with the command vgextend vg00 /dev/dsk/c2t15d0. now the new disk is the part of the VG00.
5. Use mkboot /dev/rdsk/c2t15/d0 to load the boot program in the LIF (Logical Interchange Format) area on the disk. These program includes ISL, AUTO, HPUX, PAD, LABEL.
6. Use mkboot -a "hpux" /dev/rdsk/c2t15d0 -a hpux option to add the autoboot entry to the disk.
7. Use mkboot -a "hpux -lq" /dev/rdsk/c2t15d0 -a hpux -lq to add autoboot entry to boot from the disk when quorum is lost.when hpux boots with qourum then it requires 51% of th disk space free.
8. Now check with the vgdisplay -v vg00|grep current command the order in which the lvols output are displayed.
9. Now use the same order to run the command lvextend -m 1 /dev/vg00/lvol# /dev/dsk/c2t15d0

on every lvol's in the root vg00 but execute in the same order as that of the output of the vgdisplay output.
10. Now execute lvlnboot -R /dev/vg00 it will update the root, boot and swap/dump information on the newly added PV in the VG00.
11. Check with lvlnboot -v to view the root, boot, swap/dump lvol's and check if both the pv's are displayed in the output.
12. Specify the mirrored disk as the alternate boot disk with the command setboot -a
13. Edit a file /stand/bootconf and make the following entry "l /dev/dsk/c2t15d0" so that in case of the primary disk failure HPUX can boot from the alternate disk specified in this file.
14. Boot disk mirroring for PA-RISC is done..

[Duncan Edmonstone]

Paul,

Look at the man page for vgcfgrestore... if you are sure that the contents of the backup are correct and up-to-date you can use the "-R" option to override what the kernel thinks...

HTH

Duncan

[Paul Wentland]

Thanks very much for your reply.
I remember, before I replaced the drive it was complaining about kernel mismatch.

Duncan, I have try it vgcfgrestore -R it didn't help.

Vijay, I did follow those steps up to step 9 and had an error. As well I did save lvmtab and re-created it by runnung vgscan.

I will sent my output when I get back to the office.


Mark, would removing mirror and re-imaging from scratch help? I would do that. Would I loose access to my OS, because it's a bootable drive?

Thanks for your help.

[Mark McDonald_2 · Accepted solution]

Paul you will not loose the OS, with mirrored logical volumes, the data is on both disks. An lvdisplay -v will show mirror copies = 1 and should show extents on both devices.

All I am suggesting is lvreduce -m 0 -k ....
removing the link to the broken disk. This will leave the good data on the good disk still running.

from man lvreduce:
-k This option should be used only in the special instance when you want to reduce a mirrored logical volume on a physical volume that is missing or has failed.


Once this is removed you can also vgreduce and make the system in to a 1 disk (non mirrored root) Then simply run the mirroring commands from scratch.

[Paul Wentland]

Mark, I have done the lvreduce successfully.
When I do vgcfgbackup it still complains about "vgcfgbackup: /etc/lvmtab is out of date with the running kernel:Kernel indicates 3 disks for "/dev/vg00"; /etc/lvmtab has 1 disks.
Cannot proceed with backup."

Would reboot help?

[Paul Wentland]

Following thread, especially advice from Dietmar Konermann helped me to get rid off that mismatch.
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=654700

I ended up by running those commands
#vgreduce -f vg00
#remove lvmtab
#vgscan -v

It reduced to just 1 drive.
Then followed the instruction to create a mirror from When_Good_Disks_Go_Bad_WP.pdf.

As well you have to make sure both drives a bootable and the alternate path is set in /stand/bootconf. see
http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1237221328964+28353475&threadId=1142265

This ITRC forum is awesome.

[Paul Wentland]

See my previous comment.