Root disk issue

[Sudheesh Varma VK]

I have /rp4440 server with OS version HPUX11.11 . Also root vg was mirrored.
I am facing two issues on my root VG now

1)Recently when checked the secondary disc was failed and disk was replaced. I did a pvcreate -B , mkboot and vgcfgrestore on new disk. Everythign went fine. But i am not able to sync the mirrored volume it says and " IO error " . The ioscan shows newdisk claimed. In disk info the disk is showing correct size and accessable properly.
But in vgdisplay secondary disk still shows pv not availbale.
2) Now When i do a lvlnboot -v , it is not showing primary as bootable.In case if i reboot the server will not come up at this state.Now i am not able to update lvlnboot also . It fails when i try to update.

Could any one please suggest some solution for these issues.
I am attchaching the vgdispplay and lvlnboot command outputs.

root@infsrv14:/root_home # vgdisplay -v /dev/vg00
--- Volume groups ---
VG Name /dev/vg00
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 10
Open LV 10
Max PV 16
Cur PV 2
Act PV 2
Max PE per PV 4384
VGDA 4
PE Size (Mbytes) 16
Total PE 8748
Alloc PE 3525
Free PE 5223
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg00/lvol1
LV Status available/stale
LV Size (Mbytes) 304
Current LE 19
Allocated PE 38
Used PV 2

LV Name /dev/vg00/lvol2
LV Status available/stale
LV Size (Mbytes) 8208
Current LE 513
Allocated PE 1026
Used PV 2

LV Name /dev/vg00/lvol3
LV Status available/stale
LV Size (Mbytes) 208
Current LE 13
Allocated PE 26
Used PV 2

LV Name /dev/vg00/lvol4
LV Status available/stale
LV Size (Mbytes) 208
Current LE 13
Allocated PE 26
Used PV 2

LV Name /dev/vg00/lvol5
LV Status available/stale
LV Size (Mbytes) 4000
Current LE 250
Allocated PE 500
Used PV 2

LV Name /dev/vg00/lvol6
LV Status available/stale
LV Size (Mbytes) 2912
Current LE 182
Allocated PE 364
Used PV 2

LV Name /dev/vg00/lvol7
LV Status available/stale
LV Size (Mbytes) 4000
Current LE 250
Allocated PE 500
Used PV 2

LV Name /dev/vg00/lvol8
LV Status available/stale
LV Size (Mbytes) 4112
Current LE 257
Allocated PE 514
Used PV 2

LV Name /dev/vg00/lvol9
LV Status available/stale
LV Size (Mbytes) 4096
Current LE 256
Allocated PE 512
Used PV 2

LV Name /dev/vg00/lvol10
LV Status available/syncd
LV Size (Mbytes) 304
Current LE 19
Allocated PE 19
Used PV 1


--- Physical volumes ---
PV Name /dev/dsk/c2t1d0
PV Status available
Total PE 4374
Free PE 2602
Autoswitch On

PV Name /dev/dsk/c2t0d0
PV Status unavailable
Total PE 4374
Free PE 2621
Autoswitch On
root@infsrv14:/root_home # lvdisplay -v /dev/vg00/lvol3 |more
--- Logical volumes ---
LV Name /dev/vg00/lvol3
VG Name /dev/vg00
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 208
Current LE 13
Allocated PE 26
Stripes 0
Stripe Size (Kbytes) 0
Bad block off
Allocation strict/contiguous
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c2t1d0 13 13
/dev/dsk/c2t0d0 13 13

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c2t1d0 00532 current /dev/dsk/c2t0d0 00532 stale
00001 /dev/dsk/c2t1d0 00533 current /dev/dsk/c2t0d0 00533 stale
00002 /dev/dsk/c2t1d0 00534 current /dev/dsk/c2t0d0 00534 stale
00003 /dev/dsk/c2t1d0 00535 current /dev/dsk/c2t0d0 00535 stale
00004 /dev/dsk/c2t1d0 00536 current /dev/dsk/c2t0d0 00536 stale
00005 /dev/dsk/c2t1d0 00537 current /dev/dsk/c2t0d0 00537 stale
00006 /dev/dsk/c2t1d0 00538 current /dev/dsk/c2t0d0 00538 stale
00007 /dev/dsk/c2t1d0 00539 current /dev/dsk/c2t0d0 00539 stale
00008 /dev/dsk/c2t1d0 00540 current /dev/dsk/c2t0d0 00540 current
00009 /dev/dsk/c2t1d0 00541 current /dev/dsk/c2t0d0 00541 current
00010 /dev/dsk/c2t1d0 00542 current /dev/dsk/c2t0d0 00542 current
00011 /dev/dsk/c2t1d0 00543 current /dev/dsk/c2t0d0 00543 current
00012 /dev/dsk/c2t1d0 00544 current /dev/dsk/c2t0d0 00544 stale

root@infsrv14:/root_home # lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c2t1d0 (0/1/1/0.1.0)
/dev/dsk/c2t0d0 (0/1/1/0.0.0) -- Boot Disk
No Boot Logical Volume configured
Root: lvol3 on: /dev/dsk/c2t1d0
/dev/dsk/c2t0d0
Swap: lvol2 on: /dev/dsk/c2t1d0
/dev/dsk/c2t0d0
Dump: lvol2 on: /dev/dsk/c2t1d0, 0

[Steven E. Protter]

Shalom,

rebuild mirrors following complete procedure. You have performed some but not all steps.

pvcreate -B /dev/rdsk/c1t0d0 #use real disk

mkboot -l /dev/rdsk/c1t0d0
mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c1t0d0 # use real disk


# mkboot -b /usr/sbin/diag/lif/updatediaglif -p ISL -p AUTO -p HPUX -p PAD -p LABEL /dev/rdsk/c?t?d?

If you are running 64-bit OS:

# mkboot -b /usr/sbin/diag/lif/updatediaglif2 -p ISL -p AUTO -p HPUX -p PAD -p LABEL /dev/rdsk/c?t?d?


vgextend /dev/vg00 /dev/dsk/c1t0d0 # same thing
lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c1t0d0

# real disk. repeat for other lvols

lvlnboot -r /dev/vg00/lvol3 # root fs /
lvlnboot -s /dev/vg00/lvol2 #swap
lvlnboot -d /dev/vg00/lvol2 #swap/dump
lvlnboot -b /dev/vg00/lvol1
lvlnboot -R
lvlnboot -v
setboot
setboot -a 52.1.0 # second disk

SEP

[Geoff Wild]

How to replace failed mirror of a disk

For a root disk:

After failed disk has been replaced by HP (assuming the disk is c1t5d0):

vgcfgrestore -n /dev/vg00 /dev/rdsk/c1t5d0

mkboot /dev/rdsk/c1t5d0

mkboot -a "hpux -lq (;0) /stand/vmunix" /dev/rdsk/c1t5d0

vgchange -a y /dev/vg00

lvlnboot -Rv /dev/vg00

vgsync /dev/vg00



For a non root disk:

After failed disk has been replaced by HP (assuming the disk is c1t6d0):

vgcfgrestore -n /dev/vg00 /dev/rdsk/c1t6d0

vgchange -a y /dev/vg00

lvlnboot -Rv /dev/vg00

vgsync /dev/vg00


Note:
1) If the disk is completely "dead", such as if you run ioscan and status is "no_hw" then you can hot swap the disk online.

2) However in circumstances where the disk has not fully failed please do one of the following to avoid data corruption :

a) reduce mirror before replacing the disk
b) deactivate VG before replacing the disk
c) shutdown system to replace the disk



How to mirror the root disk


Say /dev/dsk/c0t6d0 is the root disk and /dev/dsk/c3t6d0 is the disk you want to make a mirror of.

1. Create a bootable LVM disk to be used for the mirror.
pvcreate -B /dev/rdsk/c3t6d0
2. Add this disk to the current root volume group.
vgextend /dev/vg00 /dev/dsk/c3t6d0
3. Make the new disk a boot disk.
mkboot -l /dev/rdsk/c3t6d0
4. Copy the correct AUTO file into the new LIF area.
mkboot -a "hpux -lq (;0)/vmunix" /dev/rdsk/c3t6d0
5. Mirror the boot, root and primary swap logical volumes to the new
bootable disk. Ensure that all devices in vg00, such as /usr, /swap,
etc., are mirrored.
The following is an example of mirroring the boot logical volume:
lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c3t6d0
The following is an example of mirroring the primary swap logical
volume:
lvextend -m 1 /dev/vg00/lvol2 /dev/dsk/c3t6d0
The following is an example of mirroring the root logical volume:
lvextend -m 1 /dev/vg00/lvol3 /dev/dsk/c3t6d0
6. Update the boot information contained in the BDRA for the mirror
copies of boot, primary swap and root.
/usr/sbin/lvlnboot -b /dev/vg00/lvol1
/usr/sbin/lvlnboot -s /dev/vg00/lvol2
/usr/sbin/lvlnboot -r /dev/vg00/lvol3
7. Check if the BDRA is correct.
/usr/sbin/lvlnboot -R /dev/vg00
8. Verify that the mirrors were properly created.
lvlnboot -v /dev/vg00
The output of this command is shown in a display like the following:
Boot Definitions for Volume Group /dev/vg00:
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c0t6d0 (1/0/0/3/0.6.0) -- Boot Disk
/dev/dsk/c3t6d0 (1/0/1/0/0/1/1.6.0) -- Boot Disk
Boot: lvol1 on: /dev/dsk/c0t6d0
/dev/dsk/c3t6d0
Root: lvol3 on: /dev/dsk/c0t6d0
/dev/dsk/c3t6d0
Swap: lvol2 on: /dev/dsk/c0t6d0
/dev/dsk/c3t6d0
Dump: lvol2 on: /dev/dsk/c0t6d0, 0

vgdisplay -v vg00

Then lvextend for all other lvols:

lvextend -m 1 /dev/vg00/lvol4 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol5 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol6 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol7 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol8 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol9 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol10 /dev/dsk/c3t6d0
lvextend -m 1 /dev/vg00/lvol11 /dev/dsk/c3t6d0


Rgds...Geoff

[Tom Danzig]

Perhaps you have OLR. Try:

pvchange -a y /dev/dsk/c2t0d0

[Shardha]

Dear Verma,

If you have taken maketape recovery. Than I will suggest to restore the OS and them make the mirror again with the procedure give by our collegues. This will make the clean and reliable in the future.


Shardha

[tkc]

can you run dd command on the 2 disks first to confirm the disks are ok?

#dd if=/dev/rdsk/c2t1d0 of=/dev/null bs=1024k
#dd if=/dev/rdsk/c2t0d0 of=/dev/null bs=1024k

only then try on the lvm commands given.