Quorum check failed

[Titania]

After a reboot of a server, all vg's mounted perfectly except one.

messages log shows the following:

vmunix: emcp:Mpc:Error: Volume 600601605C9611007EC688BFBA3BDA11 is unbound.
vmunix: emcp:Mpc:Error: Volume 600601605C96110054EDC5DD4C4CDA11 is unbound.
emcp:Mpc:Error: Volume 600601605C96110004E7FBEE4C4CDA11 is unbound.
vmunix: LVM: VG 64 0x070000: Quorum check failed!
vmunix: LVM: VG 64 0x070000: Quorum check failed!
vmunix: LVM: VG 64 0x070000: Quorum check failed!
vmunix: LVM: VG 64 0x070000: Quorum check failed!
vmunix: LVM: VG 64 0x070000: Quorum check failed!


I ran the command:
vgchange -a y -q n /dev/vg07 and was able to re-activate de vg and mount filesystems.


any idea of what could be the cause?

[Robert-Jan Goossens_1]

Hi,

Could you post a "vgdisplay -v /dev/vg07"?

Regards,
Robert-Jan

[Ninad_1]

Hi,

It seems you have lost link to one of the disks in your VG.
Usings vgdisplay -v /dev/vg07 [ if thats the VG which gave you problem ] or strings /etc/lvmtab, check which is the other disk in vg07 which is not accessible. You can then check if the disk appears in ioscan -funC disk output. If it does not show in ioscan or shows NO_HW, then the disk is not accessible. Is this a disk on Storage SAN ? check zoning, lun security etc for that LUN.
If the disk is visible, check using diskinfo /dev/rdsk/cxtydz if it shows information about the disk.

Regards,
Ninad

[Shameer.V.A]

Hi,
Please post the following command output #vgdisplay -v /dev/vg07
#ioscan -fnC disk
#strings /etc/lvmtab

Also check with dmesg for any lost link message for the attached disks.

[Titania]

Hi,

posting vgdisplay and lvmtab entries:

vgdisplay -v vg07
--- Volume groups ---
VG Name /dev/vg07
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 4
Open LV 4
Max PV 50
Cur PV 3
Act PV 1
Max PE per PV 4096
VGDA 2
PE Size (Mbytes) 256
Total PE 199
Alloc PE 181
Free PE 18
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg07/bdtest1d
LV Status available/syncd
LV Size (Mbytes) 1024
Current LE 4
Allocated PE 4
Used PV 1

LV Name /dev/vg07/ofasdb
LV Status available/syncd
LV Size (Mbytes) 40192
Current LE 157
Allocated PE 157
Used PV 1

LV Name /dev/vg07/xsigner
LV Status available/syncd
LV Size (Mbytes) 1024
Current LE 4
Allocated PE 4
Used PV 1

LV Name /dev/vg07/accicom
LV Status available/syncd
LV Size (Mbytes) 4096
Current LE 16
Allocated PE 16
Used PV 1


--- Physical volumes ---
PV Name /dev/dsk/c20t0d2
PV Name /dev/dsk/c21t0d2 Alternate Link
PV Name /dev/dsk/c22t0d2 Alternate Link
PV Name /dev/dsk/c24t0d2 Alternate Link
PV Status available
Total PE 199
Free PE 18
Autoswitch On


******

lvmtab:

/dev/vg07
/dev/dsk/c20t0d2
/dev/dsk/c21t0d2
/dev/dsk/c22t0d2
/dev/dsk/c24t0d2

******

ioscan:

disk 51 0/0/14/1/0/4/0.41.75.0.0.0.2 sdisk CLAIMED DEVICE DGC CX700WDR5
/dev/dsk/c20t0d2 /dev/rdsk/c20t0d2


disk 52 0/0/14/1/0/4/0.41.107.0.0.0.2 sdisk CLAIMED DEVICE DGC CX700WDR5
/dev/dsk/c21t0d2 /dev/rdsk/c21t0d2


disk 92 2/0/14/1/0/4/0.51.75.0.0.0.2 sdisk CLAIMED DEVICE DGC CX700WDR5
/dev/dsk/c22t0d2 /dev/rdsk/c22t0d2


disk 91 2/0/14/1/0/4/0.51.107.0.0.0.2 sdisk CLAIMED DEVICE DGC CX700WDR5
/dev/dsk/c24t0d2 /dev/rdsk/c24t0d2


*****

diskinfo -v /dev/rdsk/c20t0d2
SCSI describe of /dev/rdsk/c20t0d2:
vendor: DGC
product id: CX700WDR5
type: direct access
size: 52428800 Kbytes
bytes per sector: 512
rev level: HP03
blocks per disk: 104857600
ISO version: 0
ECMA version: 0
ANSI version: 4
removable media: no
response format: 2
(Additional inquiry bytes: (32)68 (33)32 (34)20 (35)43 (36)4f (37)4e (38)54 (39)52 (40)4f (41)4c (42)4c (43)45 (44)52 (45)20 (46)30 (47)30 (48)30 (49)20 (50)20 (51)0 (52)0 (53)0 (54)3c (55)2 (56)60 (57)1 (58)9c (59)8 (60)c0 (61)13 (62)40 (63)0 (64)0 (65)0 (66)0 (67)0 (68)0 (69)0 (70)0 (71)0 (72)0 (73)0 (74)0 (75)0 (76)0 (77)0 (78)0 (79)0 (80)0 (81)0 (82)0 (83)0 (84)0 (85)0 (86)0 (87)0 (88)0 (89)0 (90)0 (91)2 (92)1 (93)0 (94)0 (95)32 (96)30 (97)30 (98)30 (99)31 (100)30 (101)30 (102)31 (103)31 (104)37 (105)0 (106)0 (107)0 (108)11 (109)96 (110)5c (111)0 (112)18 (113)e1 (114)5f (115)0 (116)0 (117)0 (118)0 (119)0 (120)0 (121)0 (122)0 )

[Ninad_1]

The vgdisplay output shows that there are 3 disks configured in the VG (CUR PV), whereas only 1 is available (ACT PV). so 2 of the disks in the same vg are not visible.
Can you post strings /etc/lvmconf/vg07.conf

Regards,
Ninad

[Titania]

Hi Ninad, thanx for your help.

posting vg07.conf:

strings /etc/lvmconf/vg07.conf
CONFIG01
/dev/vg07
/dev/rdsk/c21t0d2
/dev/rdsk/c20t0d2
/dev/rdsk/c24t0d2
/dev/rdsk/c22t0d2
/dev/rdsk/c21t3d2
/dev/rdsk/c22t3d2
/dev/rdsk/c24t3d2
/dev/rdsk/c21t3d3
/dev/rdsk/c22t3d3
/dev/rdsk/c24t3d3
LVMREC01
AB_f
AB_f
LVMREC01
AB_f
AB_f
LVMREC01
AB_f
AB_f
LVMREC01
AB_f
AB_f
LVMREC01
AB_f
LVMREC01
AB_f
LVMREC01
AB_f
LVMREC01
AB_f
LVMREC01
AB_f
LVMREC01
AB_f
AB_f
AB_f
VGDA0001
2E>:
VGSA0001FV

[Ninad_1 · Accepted solution]

Hi,

As you can notice from the /etc/lvmconf/vg07.conf (which is the backup taken by last vgcfgbackup) the following disks earlier available to VG are now not available.

/dev/rdsk/c21t3d2
/dev/rdsk/c22t3d2
/dev/rdsk/c24t3d2
/dev/rdsk/c21t3d3
/dev/rdsk/c22t3d3
/dev/rdsk/c24t3d3

It seems that the disk c21t3d2 has alternate links c22t3d2 and c24t3d2
Disk c21t3d3 has alternate links c22t3d3 and c24t3d3.

Pls check if ioscan shows these disks and check diskinfo.
If all disks are accessible then run the following
echo "0d8208?UU" | /usr/bin/adb /dev/rdsk/c20t0d2

echo "0d8208?UU" | /usr/bin/adb /dev/rdsk/c21t3d2

echo "0d8208?UU" | /usr/bin/adb /dev/rdsk/c21t3d3

This will display the VGID on each disk - it should be same.

Pls update after you do these activities.

Regards,
Ninad

[Ninad_1]

Hi,

Also pls check the lvdisplay -v for all the Logical volumes to see which PV it is using and if all the extents are shown current.
I have a feeling that none of the other PVs are currently being consumed by the lvols.
If this is true then you can do a vgreduce -f /dev/vg07 to remove the inaccessible and PVs.
Just do things step by step and update if you need anything

Regards,
Ninad

[Titania]

ioscan only shows disk c20t0d2 and its alternate links.

diskinfo -v /dev/rdsk/c20t0d2
SCSI describe of /dev/rdsk/c20t0d2:
vendor: DGC
product id: CX700WDR5
type: direct access
size: 52428800 Kbytes
bytes per sector: 512
rev level: HP03
blocks per disk: 104857600
ISO version: 0
ECMA version: 0
ANSI version: 4
removable media: no
response format: 2
(Additional inquiry bytes: (32)68 (33)32 (34)20 (35)43 (36)4f (37)4e (38)54 (39)52 (40)4f (41)4c (42)4c (43)45 (44)52 (45)20 (46)30 (47)30 (48)30 (49)20 (50)20 (51)0 (52)0 (53)0 (54)3c (55)2 (56)60 (57)1 (58)9c (59)8 (60)c0 (61)13 (62)40 (63)0 (64)0 (65)0 (66)0 (67)0 (68)0 (69)0 (70)0 (71)0 (72)0 (73)0 (74)0 (75)0 (76)0 (77)0 (78)0 (79)0 (80)0 (81)0 (82)0 (83)0 (84)0 (85)0 (86)0 (87)0 (88)0 (89)0 (90)0 (91)2 (92)1 (93)0 (94)0 (95)32 (96)30 (97)30 (98)30 (99)31 (100)30 (101)30 (102)31 (103)31 (104)37 (105)0 (106)0 (107)0 (108)11 (109)96 (110)5c (111)0 (112)18 (113)e1 (114)5f (115)0 (116)0 (117)0 (118)0 (119)0 (120)0 (121)0 (122)0 )


*but diskinfo for other disks shows:

diskinfo -v /dev/rdsk/c21t3d2
io_search failed: No match found.



You're right Ninad, none of that PV's are consumed by the lvols.

lvdisplay -v for all lv's shows:
--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c20t0d2 4 4


Thanks!

Could you please post correct command to perform vgreduce for these pv's?

[Ninad_1]

You can try
vgreduce /dev/vg07 /dev/dsk/c21t3d2 and so on for each disk/link for missing disks.
but, if think this will fail and you may have to use
vgreduce -f /dev/vg07
Once everything is done successfully, take a vgcfgbackup /dev/vg07

Best Luck,
Ninad

[Titania]

when trying vgreduce, got the following error:

vgreduce /dev/vg07 /dev/dsk/c21t3d2
vgreduce: Physical volume "/dev/dsk/c21t3d2" does not belong
to volume group "/dev/vg07".

[Ninad_1]

id you try the other command
vgreduce -f /dev/vg07

Regards,
Ninad

[Titania]

seems it worked:

vgreduce -f /dev/vg07
skip alternate link /dev/dsk/c21t0d2
skip alternate link /dev/dsk/c22t0d2
skip alternate link /dev/dsk/c24t0d2
PV with key 1 sucessfully deleted from vg /dev/vg07
PV with key 2 sucessfully deleted from vg /dev/vg07


vgcfgbackup /dev/vg07
Volume Group configuration for /dev/vg07 has been saved in /etc/lvmconf/vg07.conf


Thanks Ninad! with this done, in the next reboot this problem should not repeat...?

[Ninad_1]

All set and done, this should not give you any further problems.

Regards,
Ninad

[Titania]

Thank you very much!

[Titania]

found a solution following the posted replies.