topic Re: EMC VG mystery in Operating System - HP-UX

EMC VG mystery

Alan Riggs — Mon, 15 Oct 2001 18:56:38 GMT

Description:
Single Hyper VG on an EMC 8430 (one of 3).
Powerpath secondary connection exists, but is not extended into the VG as a shadow link.
Power failure on server.
After server comes up, this VG fails to activate.
vgchange -a y fails with "Quorum not present of some PV(s) are missing".
vgexport succeeds.
diskinfo on /dev/rdsk/... succeeds.
dd if=/dev/dsk/... succeeds and appears to show LVM header information.
vgcfgrestore vgsan3 /dev/dsk/... succeeds.
vgimport succeeds.
strings /etc/lvmtab shows the proper disk and VG name.
vgchange -a y fails with same error message.
If I vgexport again and vgimport using *both* primary and secondary paths to the disk, vgimport warns that the VG is suposed to have only one volume, which supports the idea that the LVM information is correct.
But I still cannot activate the VG.

Re: EMC VG mystery

Sridhar Bhaskarla — Mon, 15 Oct 2001 19:52:57 GMT

Hi Alan,

Not exactly the scenario but the similar situation happened to me when we did a business copy on XP256. We suspended the pairs and then try to import from the mirror copy. vgimport works but not vgchange. We then had to re-sync, break the mirror and it worked. So, I feel there could be a problem with the LUN itself. If you already have a mirror for this LUN, you may want to resync back and see if it works.

-Sri

Re: EMC VG mystery

Alan Riggs — Mon, 15 Oct 2001 19:56:18 GMT

Thanks. Unfortunately this volume does not have any BCV associated with it, so the restore backwards option is not available.

Re: EMC VG mystery

Animesh Chakraborty — Tue, 16 Oct 2001 01:24:37 GMT

hi,
Have you tried this one ?
vgchange -q n /dev/vgxx

Regards
Animesh

Re: EMC VG mystery

Alan Riggs — Tue, 16 Oct 2001 01:37:04 GMT

Yes. I have tried activating the VG both with and without the "-q n" flag.

Re: EMC VG mystery

Animesh Chakraborty — Tue, 16 Oct 2001 01:50:56 GMT

Hi again,
try
#insf -e

best of luck
Animesh

Re: EMC VG mystery

Alan Riggs — Tue, 16 Oct 2001 02:03:01 GMT

Ah, left that out of the original problem description. I removed the block and raw device files and recreated them with insf. No change.

Re: EMC VG mystery

George Petrides — Tue, 16 Oct 2001 06:15:29 GMT

Do you have only one physical volume in the group (excluding pvlink)? From your description, I suggest HP LVM patches and call EMC, if your EMC rocket scientist made changes to the bin file and is showing you another disk then that might be a problem.

Re: EMC VG mystery

Carlos Fernandez Riera — Tue, 16 Oct 2001 06:21:43 GMT

Try vgimport -s.

See: http://forums.itrc.hp.com/cm/QuestionAnswer/1,11866,0x9df8c6af36b7d5118ff10090279cd0f9,00.html

Re: EMC VG mystery

Alan Riggs — Tue, 16 Oct 2001 13:50:48 GMT

No changes to the BIN file.
I cannot use the -s flag for vgimport since the VG mapfile was not created with the -s flag, hence I do not know the VGID.

If someone can tell me how to capture the VGID from the disk or from the lvmconf I will try the vgimport -s.

Re: EMC VG mystery

Alan Riggs — Tue, 16 Oct 2001 13:54:42 GMT

Forgot to add:
Yes, there is only one disk (with a secondary powerpath link) in the VG.

Re: EMC VG mystery

James R. Ferguson — Tue, 16 Oct 2001 13:58:21 GMT

Hi Alan:

As far as looking at the VGID, you can get it this way:

# echo 0x2010?2X|adb /dev/dsk/cXtYdZ|expand|tr -d " "|sed "s/2010:/VGID /"

You will see something like:

VGID 11823E9D33FB2ABF

Regards!

...JRF...

Re: EMC VG mystery

Carlos Fernandez Riera — Tue, 16 Oct 2001 14:04:36 GMT

To see VGID:

Dear friend:

vgexport -s -m /tmp/vgap -p /dev/vgxx

or:

VGID is recorded in each disk:

dd if=/dev/rdsk/cxtxtx bs=1024 skip=8 count=9 | dd bs=8 skip=2 count=1 | od -x | head -1 .

Good Luck.

Re: EMC VG mystery

Alan Riggs — Tue, 16 Oct 2001 21:03:45 GMT

Thanks for the methods for capturing the VGID from disk. Unfortunately, the vgchange -a y still fails.

More information: I have also tried moving the lvmtab out of the way and recreating it with vgscan. That did not solve this problem and created another problem with a powerpath secondary, so I moved back to the old lvmtab.

Re: EMC VG mystery

Sridhar Bhaskarla — Tue, 16 Oct 2001 21:17:39 GMT

Alan,

As you were able to import the volume group the VG information is looking quite right. This should be a problem with the LUN that the EMC is not releasing it. As I mentioned earlier, it didn't work for me as long as the LUN was in suspended sync mode on XP though I could vgimport it.

Being a layman on EMC, I can't say anything on it. But I am afraid, you may have to do something on the EMC itself to get rid of the problem instead trying on the system.

Only my "0" cents.

-Sri

Re: EMC VG mystery

Carlos Fernandez Riera — Wed, 17 Oct 2001 07:37:30 GMT

Hi again:

It seems to be like a problem i had a month ago.
Working with a crashed L, vg00 (mirrored) was unrecuperable and we cannot boot after a disk crashed.

We insert a disk from other L, not mirrored to be able to boot and recover crashed vg00.

To be saved, we did a dd from not crased disk to a new disk, and also to replaced crased disk; this is we had 3 images of same disk. Then we unpluged one of these three disk. So we had a boot disk not mirrored, and a duplicated disk - i mean that VGRA was the same on both two disk-.

When we vgimport that vg for reparation ( we needed to run fsck and others), LVM said vg has ONE or THREE disks ( i dont remember exactly. but it was not true, we had only two for that vg):

More or less we had:
disk c1t2d0
disk c1t0d0

LVM-> disk c1t2d0 alternate link c1t0d0 ( remember that on a Lclass these are two different disks, but VGRA were images).

We reduce c1t0d0 because it can not be an alternate link, then LVM said not quorum.

This situation and yours seem to be equal. We had VGRA duplicated as well as you have VGRA duplicated on BCV.

I guess you must sync BCV volumes, to be only one.

After lots of trys, we recover /etc/lvmtab from a backup tape, and ... BINGO.

HTH

Re: EMC VG mystery

Carlos Fernandez Riera — Wed, 17 Oct 2001 11:44:45 GMT

Sorry. I remember BCV but it was from Sridhar?s.

Strings /etc/lvmtab must show disks involved.

run pvdisplay each disk.

And try recover /etc/lvmtab from a tape backup. On reboot a vgscan or similar runs and must overwrite the correct lvmtab. vgscan will overwrite /etc/lmvtab too.

Try recover /etc/lvmtab from a tape backup.

Re: EMC VG mystery

Alan Riggs — Wed, 17 Oct 2001 13:08:42 GMT

Sridhar,
Unfortunately, the EMC volume is not part of a disk group or associated with a BCV. There isn't any synch/suspend operation for me to clear. It is part of an M1/M2 pair, and I will ask EMC if there is any way to get a view into that relationship.

Carlos,
Strings /etc/lvmtab shows the correct information both when I have imported and exported the disk. I have recreated the file with vgscan and resored an earlier version of the file, both without effect. The warning I get when importing the VG using both paths to the disk seems to clearly indicate that the kernel is "expecting" just a single disk for this VG. I think Sridhar is on the right track that the EMC is not allowing full access to the drive for some reason, but I cannot figure out why (and neither can anyone that I have spoken to at EMC).

Re: EMC VG mystery

Amado Pichel — Thu, 18 Oct 2001 21:58:22 GMT

Alan,

Is it possible that this disk is also
presented to other servers also connected to your EMC ??

If so, Maybe it was added to another
VG on another system. If this disk was part of another VG somewhere else The VGDA would contain a VGID and you could still successfully import it but not activate it.

Now, The vgcfgrestore should fix this problem
however that would depend on whether or not the restore file for /dev/lvmconf/vgsan3.conf is valid. Is there another backup file
for vgsan3 like vgsan3.conf.old ??
If so compare the date/time stamps and try the older one.

Re: EMC VG mystery

Dennis Raudenbush — Tue, 23 Oct 2001 20:22:37 GMT

Alan, seems at this point it wouldn't hurt to vgextend the alternate disk device in the volume group and retry to activate. If it fails you could then vgreduce the primary to switch to alternate - then vgextend the primary as the alternate and retry.

johnc