corrupted mirrored disk in logical volume

[Jagadesh_2]

There was a failed disk on the server which required replacement. The onsite engineer swapped the wrong disk by mistake and then, hours later, put it back in and then swapped the faulty disk that actually needed replacement. This affected the disk management software and thus the application.


The logical volume lvol4 contains 2 disks x & y out of which y got damaged. when i issue lvdisplay it is displaying x in good state and y in stale state..

our plan is to lvreduce the damaged disk and recreate the mirror but when i issue lvreduce i am getting an error

# lvreduce -m 0 /dev/vg01/lvol4 /dev/dsk/c6t13d0
lvreduce: Warning: couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
lvreduce: Warning: couldn't query all of the physical volumes.
lvreduce: Physical volume "/dev/dsk/c6t13d0" does not belong
to volume group "/dev/vg01".
Logical volume "/dev/vg01/lvol4" is not reduced.

How to go ahead in rectifying the same. As this is a production server reboot is not possible.

[Stephen Keane]

Is /dev/dsk/c6t13d0 disk x or disk y in your description?

[Trond Haugen]

If the disk has been replaced you should restore the vg-config on it and re-sync.
vgcfgrestore -n /dev/vg04 /dev/rdsk/c6t13d0
vgchange -a y vg04

you MAY have to run 'vgsync vg04'.

Regards,
Trond

[Jagadesh_2]

disk Y is the mirrored disk.
The original disk X is safe.

[Henk Geurts · Accepted solution]

hi Jagadesh
use the lvdisplay command to find the 'pvkey'
lvdisplay -v /dev/vgo1/lvol4
.....
for example
--- Distribution of logical volume ---
PV Name LE on PV PE on PV
?????? 38 38
/dev/dsk/c2t6d0 38 38

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 ??????????????? 04228 current /dev/dsk/c2t6d0 04228 current
00001 ??????????????? 04229 current /dev/dsk/c2t6d0 04229 current
00002 ???????????????? 04230 current /dev/dsk/c2t6d0 04230 current
......



lvdisplay -v -k /dev/vg01/lvol4

output will be something like :
--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 0 04228 current 1 04228 current
00001 0 04229 current 1 04229 current
00002 0 04230 current 1 04230 current
00003 0 04231 current 1 04231 current


use the pvkey value to reduce the lvol:

lvreduce -k -m 0 /dev/vg02/lvol4
(in my example 0)

See also man lvreduce -k option.....

Regards.
Henk

[Geoff Wild]

Try to do what Trond says...

use vgcfgrestore...

As at the moment - there is no volumegroup info on that disk - that's why you are getting the error messages...

Rgds...Geoff

[Jagadesh_2]

Hi,
I tried vgcfgrestore but i am getting an error like

#vgcfgrestore -n /dev/vg01 /dev/rdsk/c6t13d0
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
Volume Group configuration has been restored to /dev/rdsk/c6t13d0

when i do pvdisplay / lvdisplay i am getting the following error

# pvdisplay -v /dev/dsk/c6t13d0 |more
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query all of the physical volumes.
pvdisplay: Couldn't retrieve the names of the physical volumes
belonging to volume group "/dev/vg01".
pvdisplay: Cannot display physical volume "/dev/dsk/c6t13d0".
udmsh02:lvmconf # lvdisplay -v /dev/vg01/lvol4 |more
lvdisplay: Warning: couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
lvdisplay: Warning: couldn't query all of the physical volumes.

[Jagadesh_2]

Hi,
I tried vgcfgrestore but i am getting an error like

#vgcfgrestore -n /dev/vg01 /dev/rdsk/c6t13d0
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
Volume Group configuration has been restored to /dev/rdsk/c6t13d0

when i do pvdisplay / lvdisplay i am getting the following error

# pvdisplay -v /dev/dsk/c6t13d0 |more
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query all of the physical volumes.
pvdisplay: Couldn't retrieve the names of the physical volumes
belonging to volume group "/dev/vg01".
pvdisplay: Cannot display physical volume "/dev/dsk/c6t13d0".

# lvdisplay -v /dev/vg01/lvol4 |more
lvdisplay: Warning: couldn't query physical volume "/dev/dsk/c6t13d0":
The specified path does not correspond to physical volume attached to
this volume group
lvdisplay: Warning: couldn't query all of the physical volumes.

[Robert-Jan Goossens]

Hi,

Henk was referring to this document.

Document description: LVM: Removing A "Ghost Disk"
Document id: LVMKBRC00006257

Europe
http://www4.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000066581312

US
http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000066581312

[Trond Haugen]

Try re-activating the vg:
vgchange -a y vg01

Regards,
Trond

[Florian Heigl (new acc)]

see if the disk is still recorded in /etc/lvmtab - but I've got a few points still unclear:

I don't see a single line of output that shows the disk as available. maybe it's the time of day, or maybe the disk is not available.

is c6t13d0 seen in ioscan -fnCdisk or is it really missing? how can it be missing - was the replacement done to another slot/scsi id?

can You access the disk using dd?

You won't be able to do the lvreduce as long as this disk sits :)

I think You'd have to either vgreduce -f, which won't work for the mirror le's, or pvcreate and vgcfgbackup and vgsync.

[Jannik]

Ok this is a triggy one.

First of all take out the faulted disk and don't put in a new. make sure the volume are running:
vgdisplay and lvdisplay.

Now remove the missing pv:
# vgreduce -f /dev/vg01
PV with key 1 sucessfully deleted from vg /dev/vg01

Move the existing lvmtab:
mv /etc/lvmtab /etc/lvmtab.old

Now create a new lvmtab:
# vgscan -v

Create a new vg01.conf
# vgcfgbackup vg01

Now you Volume consists of only one mirror and you need to remirror it agiain. So but i the new disk and do:
ioscan
insf
find the new disk and add it to the vg01
pvcreate
vgextend
lvextend -m 1 ...

look for more details at this link:
http://www5.itrc.hp.com/service/iv/node.do?admit=552267591+1108810758895+28353475&node=prodITRC%2FWW_Start%2FN1%7C16%7C11