lvmtab is out of date

[Ahmed_58]

Hi to all nice guys,
I try to extend a VG, I got a msg:

vgcfgbackup: /etc/lvmtab is out of date with the running kernel:Kernel indicates
21 disks for "/dev/fwb1vg4"; /etc/lvmtab has 17 disks.
Cannot proceed with backup.

I did mv /etc/lvmtab /etc/lvmtab.old
and run vgscan -v , but still same error when I try to add more disks!

The VG I'm trying to extend is also is part of cluster.

PS: At the end of vgscan I got the following:
Scan of Physical Volumes Complete.
*** LVMTAB has been created successfully.
*** If PV links are configured in the system.
*** Do the following to resync information on disk.
*** #1. vgchange -a y
*** #2. lvlnboot -R

appreciate ur quick answer plz.

Regards,
Ahmed

[Devender Khatana]

Hi,

The error reported is just as it is indicated. There should be 21 disks in /dev/fwb1vg4 whereas system can detect only 17 of those disks. Multiple paths can also be counted as indivisual disks. Check anything missing with

#vgdisplay -v /dev/fwb1vg4
#strings /etc/lvmtab
#ioscan -fnC disk

Attach output if you can not conclude anything from output. Any recent changes ?

HTH,
Devender

[Sivakumar TS]

Hi,

It is not recommended to run vgscan from a cluster node which is not primary.

please run the vgscan from the primary node / the node whcih owns the package.

Regards,

Siva.

[Frank de Vries]

You need to rebuild the lvmtab.

First check your cluster configuration
vgdisplay on all volumes and verify
the disks attatched.
Check with ioscan -fnC disk if
all disk are claimed.
And run vgscan.(Which you did)

Then if you kernel is still complaining,
I am afraid only a boot will reload
the correct diskdrives.

(Do you have your disks attached to a SAN,
via fibreoptics and brocades by any chance ??
or do you use IPSTORE or something similar:
We had this type of problem with a bug on your SAN software, solved now)

[Ahmed_58]

Thanks to all first,
Yes it is a promary cluster, and I checked all disks with IOSCAN vgdisplay and lvmtab all are there.

Regards,
Ahmed

[Devender Khatana]

Hi,

If all the LVOLs in the VG are shown properly in vgdisplay output and there has not been any problems you can reduce the missing LVOL from the VG using

#vgreduce -f /dev/fwb1vg4

HTH,
Devender

[Ahmed_58]

Hi devender,
the vgreduce will not solve the problem, because the disks are not shown in vgdisplay.
all the disks shown in LVMTAB are shown in VGDISPLAY.
from SAM LV display I did identfy the disks ID;s which is not in the LVMTAB.

SO....

regards,
Ahmed

[Mohanasundaram_1]

Hi Ahmed,

Have you confirmed that all the PVs for that VG are available? If so and still you get the message then vgreduce -f will indeed solve your problem.

Check the man pages of vgreduce...........

-f vg_name force reduction of missing physical volume(s)in a given volume group. This option does not require a physical volume (PV) to be supplied on the command line. vgreduce obtains the name of each
physical volume (PV) belonging to the volume group from the file /etc/lvmtab. It then reads the LVM structures from each PV and compares these with that held by the kernel to work out which PVs are missing. PVs which are missing will be candidates for removal. If all the physical extents on the missing PV are free then it will be removed from the volume group. Otherwise vgreduce will report the physical to logical extent mapping. For missing PVs, which have extents in use, you must free up all the extents by using lvreduce(1M) or lvremove(1M) and re-run vgreduce with the -f option. This option is most commonly used when the vgdisplay(1M) command shows "Cur PV" higher than "Act PV" and all of the PVs belonging to the volume group are attached. This option only works
on PVs and not on links.


I have resolved a similar issue with this option.

With regards,
Mohan.

[Ahmed_58]

Mohan, Denever & others,
Thanks all seems OK now the vgreduce & then vgscan solve the problem,... now when I try to extend the VG I'm getting this message:
-------------------------------------
vgextend fwb1vg4 /dev/dsk/c17t5d7
vgextend: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group.
-------------------------------------

A. above message came when I try to extend the VG, the disk given error above is for the alternate path which I did not issue the extend command for it after I extend the primary of it and befoe running the vgreduce/vgscan. And above message now coming when ever I extend the VG, I also try to use extend for it but getting sam msg.

B) How about the fllowing, which i got after
vgreduce:
1. save /etc/lvmtab to another file
2. remove /etc/lvmtab
3. use vgscan(1m) -v to re-create /etc/lvmtab
4. NOW use vgcfgbackup(1m) to save the LVM setup

C)And after vgscan:
Scan of Physical Volumes Complete.
*** LVMTAB has been created successfully.
*** If PV links are configured in the system.
*** Do the following to resync information on disk.
*** #1. vgchange -a y
*** #2. lvlnboot -R

Your quick reponce appreciated.

rgd,
Ahmed

[Devender Khatana]

Hi,

This is a clear cut case of some missing links. There are more than one paths defined for each PV using pvlinks but some of these are going down. If you have a close look at the vgdisplay output of ths VG you would find that some of the links has allready been lost and the disks has no redundancy.

Is it not the case? The error reported above indicates that only. Any such link once lost and then recovered after doing a vgreduce will not be included in the VG and you manually need to do vgextend to all of this alternate device files.

HTH,
Devender

[Ahmed_58]

Hi denever,
Thanks for ur replay,
Yes I try to add vgextend but failed, see below;

# strings /etc/lvmtab
/dev/fwb1vg4
/dev/dsk/c16t12d7
/dev/dsk/c16t13d0
/dev/dsk/c18t2d3
/dev/dsk/c18t4d0
/dev/dsk/c18t6d4
/dev/dsk/c25t4d0
/dev/dsk/c25t5d0
/dev/dsk/c25t5d5
/dev/dsk/c20t12d7
/dev/dsk/c20t13d0
/dev/dsk/c21t5d6
/dev/dsk/c22t2d3
/dev/dsk/c22t4d0
/dev/dsk/c22t6d4
/dev/dsk/c24t4d0
/dev/dsk/c24t5d0
/dev/dsk/c24t5d5

# vgreduce /dev/fwb1vg4 /dev/dsk/c21t5d6
vgreduce: Couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group




# vgdisplay -v fwb1vg4
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
--- Volume groups ---
VG Name /dev/fwb1vg4
VG Write Access read/write
VG Status available, exclusive
Max LV 255
Cur LV 4
Open LV 4
Max PV 60
Cur PV 8
Act PV 8
Max PE per PV 8632
VGDA 16
PE Size (Mbytes) 8
Total PE 24804
Alloc PE 20195
Free PE 4609
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
--- Logical volumes ---
LV Name /dev/fwb1vg4/lvol1
LV Status available/syncd
LV Size (Mbytes) 43136
Current LE 5392
Allocated PE 5392
Used PV 2

vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
LV Name /dev/fwb1vg4/lvol2
LV Status available/syncd
LV Size (Mbytes) 52224
Current LE 6528
Allocated PE 6528
Used PV 2

vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
LV Name /dev/fwb1vg4/lvol3
LV Status available/syncd
LV Size (Mbytes) 51200
Current LE 6400
Allocated PE 6400
Used PV 2

vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
LV Name /dev/fwb1vg4/lvol4
LV Status available/syncd
LV Size (Mbytes) 15000
Current LE 1875
Allocated PE 1875
Used PV 2


--- Physical volumes ---
PV Name /dev/dsk/c16t12d7
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c20t12d7 Alternate Link
PV Status available
Total PE 1078
Free PE 0
Autoswitch On

PV Name /dev/dsk/c16t13d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c20t13d0 Alternate Link
PV Status available
Total PE 1078
Free PE 281
Autoswitch On

PV Name /dev/dsk/c18t2d3
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c22t2d3 Alternate Link
PV Status available
Total PE 1078
Free PE 0
Autoswitch On

PV Name /dev/dsk/c18t4d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c22t4d0 Alternate Link
PV Status available
Total PE 4314
Free PE 14
Autoswitch On

PV Name /dev/dsk/c18t6d4
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c22t6d4 Alternate Link
PV Status available
Total PE 4314
Free PE 4314
Autoswitch On

PV Name /dev/dsk/c25t4d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c24t4d0 Alternate Link
PV Status available
Total PE 4314
Free PE 0
Autoswitch On

PV Name /dev/dsk/c25t5d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c24t5d0 Alternate Link
PV Status available
Total PE 4314
Free PE 0
Autoswitch On

PV Name /dev/dsk/c25t5d5
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Name /dev/dsk/c24t5d5 Alternate Link
PV Status available
Total PE 4314
Free PE 0
Autoswitch On

Ahmed.

[Devender Khatana]

Hi,

The report clearly indicated that the VG had 8 PVs with two path for each making it count 16. The same is exactly what is shown in vgdisplay output. But the one showing error is extra in /etc/lvmtab and same can be reduces by vgreduce using -f option.

#vgreduce -f /dev/fwb1vg4

HTH,
Devender

[Ahmed_58]

Devender,
Thanks again.... see below still the disk /dev/dsk/c21t5d6 appear in lvmtab;

# vgreduce -f /dev/fwb1vg4
skip alternate link /dev/dsk/c20t12d7
skip alternate link /dev/dsk/c20t13d0
vgreduce: Couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
skip alternate link /dev/dsk/c22t2d3
skip alternate link /dev/dsk/c22t4d0
skip alternate link /dev/dsk/c22t6d4
skip alternate link /dev/dsk/c24t4d0
skip alternate link /dev/dsk/c24t5d0
skip alternate link /dev/dsk/c24t5d5


# strings /etc/lvmtab
/dev/fwb1vg4
/dev/dsk/c16t12d7
/dev/dsk/c16t13d0
/dev/dsk/c18t2d3
/dev/dsk/c18t4d0
/dev/dsk/c18t6d4
/dev/dsk/c25t4d0
/dev/dsk/c25t5d0
/dev/dsk/c25t5d5
/dev/dsk/c20t12d7
/dev/dsk/c20t13d0
/dev/dsk/c21t5d6
/dev/dsk/c22t2d3
/dev/dsk/c22t4d0
/dev/dsk/c22t6d4
/dev/dsk/c24t4d0
/dev/dsk/c24t5d0
/dev/dsk/c24t5d5

Regards,
Ahmed

[Devender Khatana]

Hi,

That is OK that this disk in visible there in lvmtab under that VG. Can you seen through diskinfo if the disk is correct.

I think it should be fine. But the reason for that problem is probably because the disk has been replaced recently without notifying the system. i.e. no vgcfgrestore was done on the system.

If a disk has gone faulty and you do a vgreduce -f it will be removed from lvmtab but it is not happening here as the disk is not missing and is repondling properly and the information on disk do not show that this disk belongs to this VG.

Was it replaced recently if not then it was overwritten by something like dd.

Find out the disk layout by

#pvdisplay -v /dev/dsk/c21t5d6

Also the simplest solution that you could apply here was just remove this disk physically and then do a

#vgreduce -f /dev/fwb1vg4


HTH,
Devender

[Ahmed_58]

Denever,
If you look at my previous msg I did vgreduce -f /dev/fwb1vg4 but still when display lvmtab shows it is there?

By the way the primary path is not shown in the lvmtab.

Regards,
Ahmed

[Devender Khatana]

Hi,

In earlier response I have asked to use vgreduce -f after physically removing the disk from system or array.

HTH,
Devender

[Ahmed_58]

Denever,
This disk needed, we can't remove it, I needed to be added to VG properly with both primary/alt.

Regards,
Ahmed

[Devender Khatana]

Hi,

Removing mean remove it than do a vgreduce with -f flag as mentioned earlier and then attach it back at the same path and use vgextend to both the paths one by one.

Also you can try

#vgcfgrestore -n /dev/fwb1vg4 /dev/rdsk/c21t5d6

If vgcfgrestore is successul then you would not require the earlier steps and at the end just extend the vg to the alternate device for this disk.

HTH,
Devender

[Ahmed_58]

Hi
This is a EMC disks, removing need to call Engineer, I try below: it seems it is the old confg file;

# gcfgrestore /dev/fwb1vg4 /dev/rdsk/c21t5d6
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c17t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c21t5d6":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c17t5d7":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c21t5d7":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c17t6d0":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c21t6d0":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgrestore: Mismatch between the backup file and the running kernel:
Kernel indicates 16 disks for "/dev/fwb1vg4"; /etc/lvmconf/fwb1vg4.conf indicate
s 22 disks.
Cannot proceed with the restoration. Deactivate the Volume Group and try again. try

[Ahmed_58]

vgscan -va sort my problem ... thanks.