Unable to activate VG

[Karthik S S]

For some strage reasons I am unable to activate the VG on one of the K-Class system. All the disks are visible in ioscan and entries are there in lvmtab. Pl. help.

Thanks,
Karthik S S

#########################################################
vgchange output
#########################################################

# vgchange vg02

vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c3t8d0":
Cross-device link
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c3t9d0":
Cross-device link
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c3t10d0":
Cross-device link
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c3t11d0":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c3t8d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query physical volume "/dev/dsk/c3t9d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query physical volume "/dev/dsk/c3t10d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query physical volume "/dev/dsk/c3t11d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "vg02":
Quorum not present, or some physical volume(s) are missing.

#########################################################
pvdisplay output
#########################################################

# pvdisplay /dev/dsk/c3t8d0

pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t8d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t9d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t10d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t11d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query all of the physical volumes.
pvdisplay: Couldn't retrieve the names of the physical volumes
belonging to volume group "/dev/vg02".
pvdisplay: Cannot display physical volume "/dev/dsk/c3t8d0".

#########################################################
diskinfo output
#########################################################

# diskinfo /dev/rdsk/c3t8d0

SCSI describe of /dev/rdsk/c3t8d0:
vendor: SEAGATE
product id: ST34572WC
type: direct access
size: 4194157 Kbytes
bytes per sector: 512

#########################################################
lvmtab output
#########################################################

# strings /etc/lvmtab

/dev/vg00
/dev/dsk/c2t5d0
/dev/vg01
/dev/dsk/c0t5d1
/dev/dsk/c2t4d0
/dev/dsk/c2t8d0
/dev/dsk/c3t12d0
/dev/dsk/c1t5d1
/dev/dsk/c3t13d0
/dev/dsk/c3t14d0
/dev/dsk/c3t15d0
/dev/vg02
/dev/dsk/c3t8d0
/dev/dsk/c3t9d0
/dev/dsk/c3t10d0
/dev/dsk/c3t11d0

[Dietmar Konermann · Accepted solution]

Karthik,

a cross-device link error is typically issued if the PVID on disk does not match the one stored in lvmtab. A corrupted or wiped-out LVM header may be also a potential reson.

Run this to see the VGIDs in lvmtab:

strings -t d /etc/lvmtab |
grep /dev |
grep -v /dev/dsk |
while read offset path; do
xd -An -j$(($offset+1024)) -N8 -tuL /etc/lvmtab |
read vgid1 vgid2
echo $path $vgid1-$vgid2
done

Run this to see a VGID on a given disk:

xd -An -j8208 -N8 -tuL /dev/dsk/cXtYdZ

Best regards...
Dietmar.

[Karthik S S]

Hi Dietmar,

Thanks a lot. By running the commands you suggested I get this,

# grep -v /dev/dsk|while read offset path;do
> xd -An -j$(($offset+1024)) -N8 -tuL /etc/lvmtab |
> read vgid1 vgid2
> echo $path $vgid1-$vgid2
> done
/dev/vg00 1666441341-1070415279
/dev/vg01 1666441341-1070572981
/dev/vg02 1666441341-1073859961
/dev/vg03 1666441341-1074803278
/dev/vg04 1666441341-1074804443

----------------------------------------------

# xd -An -j8208 -N8 -tuL /dev/dsk/c3t8d0
0 0

# xd -An -j8208 -N8 -tuL /dev/dsk/c3t9d0
1667441341 1075318579

# xd -An -j8208 -N8 -tuL /dev/dsk/c3t10d0
0 0

# xd -An -j8208 -N8 -tuL /dev/dsk/c3t11d0
0 0

#
-------------------------------------------

Now how do I proceed with this info.? Doing a vgexport/import and or vgcfgrestore help me resolve the problem??

-Karthik S S

[T G Manikandan]

try re-creating the lvmtab

# mv /etc/lvmtab /etc/lvmtab.old
# vgscan -va
# vgchange -a y /dev/vg02

[Karthik S S]

I tried recreating lvmtab. But it doesn't recognize vg02.

-Karthik S S

[Dietmar Konermann]

Karthik,

some of your on-disk VGIDs are 0... this does not look good, 0 means 'not part of any VG'. What happened to theses disks? Were they pvcreate'd? Or accidently erased/overwritten somehow?

Try to restore the headers using vgcfgrestore. But it could be possible that user data is wiped-out also.

Best regards...
Dietmar.

[Chris Watkins_1]

Does it list out those disks together, though?
If so, you can probably vgexport and reimport and be golden.

# vgexport vg02
#vgimport vg02 /dev/dsk/c3t8d0 /dev/dsk/c3t9d0 /dev/dsk/c3t10d0 /dev/dsk/c3t11d0

If that doesn't work, make sure your vg minor numbers are different:
# ll /dev/vg*/group


Did any configuration on the machine change?
New hardware, lun presentation from SAN, etc...?

Are the disks shared with another machine?

[Karthik S S]

I am not sure if someboy tried pvcreate on these disks. The problem has landed to me after many hops :-( .... vgimport didn't work either.

# vgimport vg02 /dev/dsk/c3t8d0 /dev/dsk/c3t9d0 /dev/dsk/c3t10d0 /dev/dsk/c3t11d0
Physical Volume "/dev/dsk/c3t8d0" is not part of a Volume Group


-Karthik S S

[Karthik S S]

I did a vgcfgrestore and was able to activate the VG.

# vgcfgrestore -n /dev/vg02 /dev/rdsk/c3t8d0
Volume Group configuration has been restored to /dev/rdsk/c3t8d0
# vgcfgrestore -n /dev/vg02 /dev/rdsk/c3t9d0
Volume Group configuration has been restored to /dev/rdsk/c3t9d0
# vgcfgrestore -n /dev/vg02 /dev/rdsk/c3t10d0
Volume Group configuration has been restored to /dev/rdsk/c3t10d0
# vgcfgrestore -n /dev/vg02 /dev/rdsk/c3t11d0
Volume Group configuration has been restored to /dev/rdsk/c3t11d0
#
---------------------------

Vgdisplay also looks fine.

# vgdisplay vg02
--- Volume groups ---
VG Name /dev/vg02
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 0
Max PV 16
Cur PV 4
Act PV 4
Max PE per PV 1023
VGDA 8
PE Size (Mbytes) 4
Total PE 4092
Alloc PE 4000
Free PE 92
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

----------------------------------------

Earlier only one LV was configured under this VG (lvol1). But before this as I did a vgexport/vgimport, the lvol device file is missing in the /dev/vg02 directory. Can I create it using mknod command? What is the syntax?

# ll /dev/vg02
total 0
crw-rw-rw- 1 root sys 64 0x020000 Feb 4 16:50 group
#

Thanks,
Karthik S S

[Karthik S S]

Did any configuration on the machine change?
New hardware, lun presentation from SAN, etc...?
-- I didn't do any changes. But it is possible that somebody might have tried before the case came to me.

Are the disks shared with another machine?
-No. these are JBODs directly attached to this system.

Thanks
Karthik S S

[Mark Grant]

yes you can just mknod it. If you are in the directory containing "group" already.

It will be "mknod lvol2 b 64 0x020002" and then don't forget the raw device too

"mknod rlvol2 c 64 0x020002"

[Karthik S S]

Instead of mknod, I did a

vgexport
vgscan
and
vgimport


I was able to recover the VG. Thanks a lot for all the responses :-)

-Karthik S S