Reduce no. of disks (mirror copies) in root volume group

[Tomas Moravcik]

Hi all,

This applies to PA-RISC server running HP-UX 11.23.

The root volume group (vg00) contained 2 disks, one of them failed so I had to remove it. The problem is, none of the recommended commands work here. Here's the situation:

# vgdisplay -v
--- Volume groups ---
VG Name /dev/vg00
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 11
Open LV 11
Max PV 16
Cur PV 2
Act PV 1
Max PE per PV 4384
VGDA 2
PE Size (Mbytes) 16
Total PE 4340
Alloc PE 2677
Free PE 1663
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg00/lvol1
LV Status available/stale
LV Size (Mbytes) 1792
Current LE 112
Allocated PE 224
Used PV 1

LV Name /dev/vg00/lvol2
LV Status available/stale
LV Size (Mbytes) 4096
Current LE 256
Allocated PE 512
Used PV 1

LV Name /dev/vg00/lvol3
LV Status available/stale
LV Size (Mbytes) 1264
Current LE 79
Allocated PE 158
Used PV 1

LV Name /dev/vg00/lvol4
LV Status available/stale
LV Size (Mbytes) 512
Current LE 32
Allocated PE 64
Used PV 1

LV Name /dev/vg00/lvol5
LV Status available/stale
LV Size (Mbytes) 5120
Current LE 320
Allocated PE 640
Used PV 1

LV Name /dev/vg00/lvol6
LV Status available/stale
LV Size (Mbytes) 7008
Current LE 438
Allocated PE 876
Used PV 1

LV Name /dev/vg00/lvol7
LV Status available/stale
LV Size (Mbytes) 5088
Current LE 318
Allocated PE 636
Used PV 1

LV Name /dev/vg00/lvol8
LV Status available/stale
LV Size (Mbytes) 5664
Current LE 354
Allocated PE 708
Used PV 1

LV Name /dev/vg00/lvol9
LV Status available/stale
LV Size (Mbytes) 2048
Current LE 128
Allocated PE 256
Used PV 1

LV Name /dev/vg00/lvol10
LV Status available/stale
LV Size (Mbytes) 5120
Current LE 320
Allocated PE 640
Used PV 1

LV Name /dev/vg00/lvol11
LV Status available/stale
LV Size (Mbytes) 5120
Current LE 320
Allocated PE 640
Used PV 1


--- Physical volumes ---
PV Name /dev/dsk/c2t0d0
PV Status available
Total PE 4340
Free PE 1663
Autoswitch On
Proactive Polling On


# lvdisplay -v -k /dev/vg00/lvol1
--- Logical volumes ---
LV Name /dev/vg00/lvol1
VG Name /dev/vg00
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 1792
Current LE 112
Allocated PE 224
Stripes 0
Stripe Size (Kbytes) 0
Bad block off
Allocation strict/contiguous
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c2t0d0 112 112

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 0 00000 stale 1 00000 stale
--------------------------------------------
00111 0 00111 stale 1 00111 stale

#

If I try to reduce the mirror copies I get this error:

# lvreduce -m 0 -k /dev/vg00/lvol1 0
Physical extents on remaining physical volumes are stale or
Remaining physical volumes are not responding.
lvreduce: The LVM device driver failed to reduce mirrors on
the logical volume "/dev/vg00/lvol1".
#

Could you advice?

Thanks,
Tomas

[Torsten.]

There is absolutely no need to reduce the mirror!

You already rebooted the system (why?).

Now get the other disk:

# strings /etc/lvmtab

Your working disk is /dev/dsk/c2t0d0.

Let's assume the failed disk is

/dev/dsk/c2t1d0

Correct steps would be

# pvchange -a n /dev/dsk/c2t1d0

- swap the disk

# vgcfgrestore -n vg00 /dev/dsk/c2t0d0

# pvchange -a y /dev/dsk/c2t1d0

(# vgchange -a y vg00)

- make it bootable
- sync


For complete steps see

http://docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad.pdf

[TTr]

Are you planning on replacing the failed disk or you will run with a single disk?

[Torsten.]

I don't know if you have tried to remove the working disk by accident, but now it looks like the extends on both disks are bad:

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 0 00000 stale 1 00000 stale
--------------------------------------------
00111 0 00111 stale 1 00111 stale


You should be prepared for an ignite recovery ...

[Tim Nelson]

I think Torsten's last comment is valid.

Beware that the disk that reports as stale may not be the bad disk. The good one could be stale because it cannot read the bad one's extent.

Always check with the following. Whichever disk fails the dd is the bad one. Either way you will need to recover this system via ignite.

dd if=/dev/rdsk/cxtxdx of=/dev/null

[Ganesan R]

Hi,

If you removed the working disk by accident,
put back both the disks and try booting from both primary and secondary before do anything.

If you are able to boot from any one then you are lucky enough to recover the system.

Once you are booted from one , remove the other disks and follow the steps to restore mirror.

let me know if you need the steps to remirror.

[Tomas Moravcik]

@Thorsten:

You already rebooted the system (why?). - The system crashed after disk failure.

So, the lvmtab contains only one disk - c2t0d0:

# strings /etc/lvmtab
/dev/vg00
Hb;B
/dev/dsk/c2t0d0
#

which is strange because of this (vgdisplay output):

Cur PV 2
Act PV 1

# vgcfgrestore -n vg00 /dev/dsk/c2t0d0
vgcfgrestore: Cannot restore Physical Volume "/dev/dsk/c2t0d0",
Detach the PV or detactivate the VG, before restoring the PV.
#

This doesn't work for root volume (I think).

So I created new PV (pvcreate -fB /dev/rdsk/c2t1d0) and then added to vg00 (vgextend /dev/vg00 /dev/dsk/c2t1d0) and when syncing this error occured:

# lvsync -T /dev/vg00/lvol*
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol1".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol10".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol11".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol2".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol3".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol4".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol5".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol6".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol7".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol8".
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol9".
#

@Ganesan R

Could you please explain the steps for re-mirrorig?


Thank you all.

[Torsten.]

If you have stale extends on your single root disk you are out of luck - you need to look for your recently made backup.


Now about this:

# vgcfgrestore -n vg00 /dev/dsk/c2t0d0
vgcfgrestore: Cannot restore Physical Volume "/dev/dsk/c2t0d0",
Detach the PV or detactivate the VG, before restoring the PV.
#

This doesn't work for root volume (I think).



I hope you know this already:

http://docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad.pdf


The disk needs to be deactivated before removal or vgcfgrestore:

# pvchange -a n /dev/dsk/c2t0d0
# vgcfgrestore -n vg00 /dev/dsk/c2t0d0
# pvchange -a y /dev/dsk/c2t0d0

[Torsten.]

If dev/rdsk/c2t1d0 was your failed disk you need to do this with dev/rdsk/c2t1d0, of course.

[Ganesan R]

Hi,

You are trying to restore on the available disk.
>>>>>>>>
# vgcfgrestore -n vg00 /dev/dsk/c2t0d0
vgcfgrestore: Cannot restore Physical Volume "/dev/dsk/c2t0d0",
Detach the PV or detactivate the VG, before restoring the PV.
#
<<<<<<<

find out the other disk from conf file and restore the lvm headers to that disk.

#vgcfgrestore -n /dev/vg00 -l

it will list the the other disk.

Restore the configuration on alt disk.
#vgcfgrestore -n /dev/vg00
Reactivate the VG
#vgchange -a y vg00
Sync the VG
#vgsync

[Sandeep_Chaudhary]

pvchange -a n /dev/dsk/c2t1d0
vgcfgrestore -n vg00 /dev/dsk/c2t1d0
pvchange -a y /dev/dsk/c2t1d0
vgsync vg00

lnlnboot -R

lvlnboot -v

Try this and

[Ganesan R]

Hi,

Herewith I have attched the steps for doing the remirror as per your request.

Refer the steps.

[Tomas Moravcik]

Thanks guys, I have to restore from backup.