Impact on playing with vg....

[Yan Berube_1]

I'm actually expecting some problem on one of my server, who is 9000/800/A500, running with hp-ux 11.00 and after a reboot my vg01 is not anymore available... as I have a lot of important data on it, don't want to do something wrong.

the problem I have is that vg01 is not activated. vgchange is giving me an error meg : Cross-device link... same if I do vgchange -a y -q n vg01... ioscan and ioscan told me that the disk is there and working. Don't really know what I can do now... Can you help me to not lost my data, plz ?

[Robert-Jan Goossens_1]

Hi,

Could be the /etc/lmvtab is corrupt ( one disk being recorded by two volume groups.

# strings /etc/lvmtab

If this is the case.

# mv /etc/lvmtab /etc/lvmtab.old

# vgscan -v

# vgchange -a y /dev/vg01

Hope this helps,
Robert-Jan

[saju_2]

Hi

Have you done a disk-disk copy before the reboot? I faced this error when the source disk and destination disk had the same VG info (VGID, PVID) etc.

Regards
CS

[Mel Burslan]

try to restore the LVM headers first

# vgcfgrestore -n /dev/vg01 /dev/rdsk/cXtXdX
(repeat as many times as necessary to cover all disks belonging to vg01)

If it still is reporting problems, try rebuilding the lvmtab

# mv /etc/lvmtab /etc/lvmtab.SAV
# vgscan -av

HTH

[Yan Berube_1]

Forget to tell that vg01 is in fact a VA 7110... So, when we do a vgcfgrestore are we doing it only once ?

[Devender Khatana]

Hi,

Can you post output of "strings /etc/lvmtab" and "ioscan -fnC disk" to know what are the device files before change and what is this now. If you have allready move lvmtab to some alternate location then use strings on that name only as the contents of old lvmtab will be helpful to know the path of old disks in the VG.

HTH,
Devender

[Yan Berube_1]

I did nothing until now... here is the result of few cmd line on the server:

# > strings /etc/lvmtab
/dev/vg00
/dev/dsk/c1t15d0
/dev/dsk/c3t15d0
/dev/vg01
/dev/dsk/c4t0d1
# > vgchange -a y /dev/vg01
vgchange: Warning: Couldn't attach to the volume group physical
volume "/dev/dsk/c4t0d1":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c4t0d1":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "/dev/vg01":
Quorum not present, or some physical volume(s) are missing.

# > vgscan -v
vgscan: Warning: couldn't query physical volume "/dev/dsk/c4t0d1":
The specified path does not correspond to physical volume attached to
this volume group
vgscan: Warning: couldn't query all of the physical volumes.
vgscan: The physical volume "/dev/dsk/c1t15d0" is already recorded in
the "/etc/lvmtab" file.
vgscan: The physical volume "/dev/dsk/c3t15d0" is already recorded in
the "/etc/lvmtab" file.
vgscan: The physical volume "/dev/dsk/c4t0d1" is already recorded in
the "/etc/lvmtab" file.
Couldn't stat physical volume "/dev/dsk/c0t2d0":
Invalid argument
Physical Volume "/dev/dsk/c4t0d0" contains no LVM information
Physical Volume "/dev/dsk/c6t0d0" contains no LVM information


/dev/vg00
/dev/dsk/c1t15d0
/dev/dsk/c3t15d0


Following Physical Volumes belong to one Volume Group.
Unable to match these Physical Volumes to a Volume Group.
Use the vgimport command to complete the process.
/dev/dsk/c4t0d1
/dev/dsk/c6t0d1

The Volume Group /dev/vg01 was not matched with any Physical Volumes.
Scan of Physical Volumes Complete.

[Devender Khatana]

Hi,

Could you also attach the output of "ioscan -fnC disk" to know what are the current device files for these disks. I assume that the device file for /dev/dsk/c4t0d1 has changed to /dev/dsk/c6t0d1 which is causing this. Or your disk in accessible from both paths.

Can you try
#vgexport /dev/vg01

To remove current config of the VG from the system & then try reimporting the VG from the new device file of the disk which is currently visible to system.

#mkdir /dev/vg01
#mknod /dev/vg01/group c 64 0x0?0000 (? should be unique identify by "ll /dev/*/group")
#vgimport /dev/vg01 /dev/dsk/c6t0d1

I assume some other changes at back end which is causing these device file changes. A reboot in any case can not cause this.

HTH,
Devender

[Yan Berube_1]

Here is the output of the ioscan -nfC disk:

Class I H/W Path Driver S/W State H/W Type Description
==================================================================================
disk 2 0/0/1/0.2.0 sdisk CLAIMED DEVICE HP DVD-ROM 305
/dev/dsk/c0t2d0 /dev/rdsk/c0t2d0
disk 0 0/0/1/1.15.0 sdisk CLAIMED DEVICE HP 36.4GMAS3367NC
/dev/dsk/c1t15d0 /dev/rdsk/c1t15d0
disk 1 0/0/2/1.15.0 sdisk CLAIMED DEVICE HP 36.4GMAS3367NC
/dev/dsk/c3t15d0 /dev/rdsk/c3t15d0
disk 3 0/4/0/0.8.0.108.0.0.0 sdisk CLAIMED DEVICE HP A6189B
/dev/dsk/c4t0d0 /dev/rdsk/c4t0d0
disk 5 0/4/0/0.8.0.108.0.0.1 sdisk CLAIMED DEVICE HP A6189B
/dev/dsk/c4t0d1 /dev/rdsk/c4t0d1
disk 4 0/6/2/0.8.0.110.0.0.0 sdisk CLAIMED DEVICE HP A6189B
/dev/dsk/c6t0d0 /dev/rdsk/c6t0d0
disk 6 0/6/2/0.8.0.110.0.0.1 sdisk CLAIMED DEVICE HP A6189B
/dev/dsk/c6t0d1 /dev/rdsk/c6t0d1


Please note that /dev/dsk/c4t0d1 is normally the primary path to access the Hp Virtual Array 7110 and the /dev/dsk/c6t0d1 is the alternate path... Can you confirm by doing the vgexport /dev/vg01, i will not lost any data ? Sorry, but don't want to lost something...

Many Thanks for your help

[Devender Khatana · Accepted solution]

Hi,

vgexport & import will not loose data.

Also please specify both the device files while importing the VG i.e.

#vgimport /dev/vg01 /dev/dsk/c4t0d1 /dev/dsk/c6t0d1

===================================
Physical Volume "/dev/dsk/c4t0d0" contains no LVM information
Physical Volume "/dev/dsk/c6t0d0" contains no LVM information


/dev/vg00
/dev/dsk/c1t15d0
/dev/dsk/c3t15d0


Following Physical Volumes belong to one Volume Group.
Unable to match these Physical Volumes to a Volume Group.
Use the vgimport command to complete the process.
/dev/dsk/c4t0d1
/dev/dsk/c6t0d1
=======================================

From you earlier post it seems /dev/dsk/c4t0d0 is the primary path & /dev/dsk/c6t0d0 is the alternate path to access VA, not /dev/dsk/c4t0d1 & /dev/dsk/c6t0d1 as posted by you. This is also confirmed by contents of your old lvmtab which states VG to reside on /dev/dsk/c4t0d1.

If you are right then your balance disks do not have any LVM information on them. Clarify this before proceeding further. Also find backend chnages done.

HTH,
Devender

[Yan Berube_1]

To clarify, /dev/dsk/c4t0d0 is the primary path for the LUN 0 on the HP VA 7110, it's a small LUM of 10 meg. /dev/dsk/c6t0d0 is his alternate path. No LVM information in this LUN.

Here is what I found in the log of the server, about vg operation, these cmd was execute in February. Will have to investigate to found who did that on this system, but it can be useful for you to help me.

bdf
vgdisplay -v vg01
diskinfo -v /dev/rdsk/c6t0d1
diskinfo -v /dev/rdsk/c4t0d1
armdsp -i
ioscan -fknCdisk
strings /etc/lvmtab
pvdisplay -v /dev/dsk/c6t0d1pvdisplay -v /dev/dsk/c6t0d1|more
pvdisplay -v /dev/dsk/c4t0d1|more
vgreduce vg01 /dev/dsk/c6t0d1^
vgreduce vg01 /dev/dsk/c6t0d1
vgdisplay -v vg01
cd /etc/lvmconf
ll
strings vg01.conf
cp vg01.conf vg01.conf.10022005
vgcfgbackup vg01
vgcfgrestore -n vg01 /dev/rdsk/c4t0d1
vgextend vg01 /dev/dsk/c6t0d1
vgdisplay -v vg01
^B^Wvgcfgrestore -n vg01 /dev/rdsk/c4t0d1
vgdisplay -v vg01
vgreduce vg01 /dev/dsk/c6t0d1
vgdisplay -v vg01
vgextend vg01 /dev/dsk/c6t0d1
vgdisplay -v vg01
ioscan -fknCdisk
diskinfo -v /dev/rdsk/c4t0d0
diskinfo -v /dev/rdsk/c4t0d1
vgcfgrestore -n vg01 /dev/rdsk/c4t0d1
vgdisplay -v vg01
ll
strings vg01.conf.old
strings vg01.conf.10022005
more vg01.conf
armdsp -i

[Yan Berube_1]

So, /dev/dsk/c4t0d1 is referring to the LUN 1 on the VA 7110 and it is the primary path for the vg01, c6t0d1 is the alternate.

[Devender Khatana]

Yan,

Go ahead with vgexport and vgimport. It is safe and do not destroy data in any case. To confirm the steps are.

1.Note down minor no. from /dev/vg01/group file. by doing "ll /dev/vg01/group" it should be 0x010000 for vg01 generally.

2.#vgexport /dev/vg01

3.#mkdir /dev/vg01; mknod /dev/vg01/group c 64 0x010000 ( Or other minor no. caught from step 1)

4.#vgimport /dev/vg01 /dev/dsk/c4t0d1 /dev/dsk/c6t0d1

5. vgchange -a y /dev/vg01

6. strings /etc/lvmtab

7. vgdisplay -v /dev/vg01 |more

8. mountall

9. bdf

Steps 6-9 are only for confirming the results.

Do not forget to post results, assigning points and closing thread after resolution.

Regards,
Devender

[Yan Berube_1]

#> ll /dev/vg01/group
crw-r--r-- 1 root sys 64 0x010000 Nov 12 2003 /dev/vg01/group
#> vgexport /dev/vg01
#> mkdir /dev/vg01; mknod /dev/vg01/group c 64 0x010000
#> ll /dev/vg01
total 0
crw-rw-rw- 1 root sys 64 0x010000 Jul 12 23:32 group
#> vgimport /dev/vg01 /dev/dsk/c4t0d1 /dev/dsk/c6t0d1
vgimport: Warning: Volume Group contains "1" PVs, "2" specified. Continuing.
vgimport: Quorum not present, or some physical volume(s) are missing.
Warning: A backup of this volume group may not exist on this machine.
Please remember to take a backup using the vgcfgbackup command after activating the volume group.
#> vgchange -a y /dev/vg01
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c4t0d1":
Cross-device link
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c6t0d1":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c4t0d1":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query physical volume "/dev/dsk/c6t0d1":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "/dev/vg01":
Quorum not present, or some physical volume(s) are missing.

#> strings /etc/lvmtab
/dev/vg00
/dev/dsk/c1t15d0
/dev/dsk/c3t15d0
/dev/vg01
/dev/dsk/c4t0d1
/dev/dsk/c6t0d1

Still the same problem... Don't understand where is the problem... Any other idea ?

[Devender Khatana]

Hi,

Try

#vgchange -a y -q n /dev/vg01

The problem is only that instead of two links to the same disk it is sensing two device files as two seperate disks.

If vgchange is still not done try

#vgreduce -f /dev/vg01

#strings /etc/lvmtab
Now retry with vgchange without -q option.
#vgchange -a y /dev/vg01

If activated try mounting volumes & check everything. If everything is fine extend vg to other device file.
#vgextend /dev/vg01 /dev/dsk/c6t1d0

The other device file should be visible as alternate link in the output of "vgdisplay -v /dev/vg01 |more". But I think it will not.
You can leave without extending also but then it will be accessible through one path only.

Post results of all the commands in the sequence.

HTH,
Devender

[Yan Berube_1]

#> vgchange -a y -q n /dev/vg01
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c4t0d1":
Cross-device link
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c6t0d1":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c4t0d1":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query physical volume "/dev/dsk/c6t0d1":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "/dev/vg01":
Quorum not present, or some physical volume(s) are missing.

#> vgreduce -f /dev/vg01
vgreduce: The volume group "/dev/vg01" is not active. Only an active
volume group can be extended.
#> strings /etc/lvmtab
/dev/vg00
/dev/dsk/c1t15d0
/dev/dsk/c3t15d0
/dev/vg01
/dev/dsk/c4t0d1
/dev/dsk/c6t0d1

no more success...

[Devender Khatana]

Yan,

Please retry the export/import steps but while doing a vgimport only specify one physical disk i.e. /dev/dsk/c4t0d1.

Regards,
Devender

[saju_2]

Hi yan/devender

I have already posted a similar reply but again i suspect the same problem.

In our server we do a BC(Business copy) form XP1024. We dont generally do a vgchgid on the SVOL (mirror copy) while doing a vgimport . So the next time when i reboot the server this same error we used to get. Why cant try a vgchgid /dev/rdsk/cxtxdx /dev/rdsk/cytydy etc... Pls mention all the disks in the VG while doing a vgchgid command


regards
CS

[Yan Berube_1]

same result... no luck...

[Devender Khatana]

Yan,

It should not be. Try same with other device file. Post the results in sequence. If still not successful then try removing one path and then retry the operation with one device that is available.

Saju,

Here there is no BC volumes these are different paths to the same disk.

HTH,
Devender

[andrew medhurst]

Hi Guys just a quick note to ask if the alternate path is through a seperate controller card in the server and if so have you checked the card because it does seem to be talking about the alternate path which it can no longer see and therefore that would also give the qourum error.
did the card go down after the reboot i have had same happen to me before.
also have you run a vgdispaly -v and a pvdisplay -v on both paths
regards
andrew

[Yan Berube_1]

Hi all,

Many thanks for your reply... After few discussion with HP expert, we conclude that the lvm configuation was corrupted. And HP recommend to rebuild the vg01 and restore the data.

We are still trying to avoid that, so, if you have any other idea... we have nothing to lost to try it...

Regards, Yan

[Thayanidhi]

I suspect the alternate path is having a intermintent problem. Though ioscan gives All disks CLAIMED, check the syslog.log for device related errors!

Try to do vgexport/import only with one PV path. To activate th vg use "vgchange -a y -q n "

Regds
TT
You can also use fcmsutil to find both HBA are active.

[David Ritchie]

Couldn't you just simply

mv /etc/lvmtab /etc/lvmtab.old
vgscan -a -v

Then use
strings /etc/lvmtab
to see if the disk devices look sane.... you then should be able to simply
vgchange -a y /dev/vg01
to activate the volume groups.

This effectively does the same thing as the
vgexport without having to have to have
everything initially consistant. Up to the point where you vgchange, it is my understanding you can get back to where you were by simply copying back the /etc/lvmtab.old to /etc/lvmtab.

I have attached source for a program you can
compile and then use to dump vgids associated with a particular disk.


It will give output that looks like this:


root on ftpsvr [/der]: ./dumpvgid /dev/dsk/c15t1d1
/dev/dsk/c15t1d1 LVMREC01 0xb78043854176ed54 0xb78043854176ed57

The first string is the VGID the second is the PVID. Why vgdisplay and pvdisplay don't show these is a mystery to me....

--- Dave Ritchie

[David Ritchie]

just another point - you should see consistant VGID and PVID's on any primary or alternate links (which is why I attached the program to look at the VGID/PVID's). I suspect that if you look at these for the volume group, the PVID's don't match between the primary and alternate links. Also, you may be linked into another VG's disks... this is all straighten out by the vgscan, at least to my understanding... correct me if I am wrong....

-- Dave Ritchie