MC/SG thinks root VG is a clustered VG!!

[Thomas J. Harrold]

I get the following errors when I run a cmcheckconf:

root@pfs1:[/etc/cmcluster]# cmcheckconf -C /etc/cmcluster/pfs1_clust.conf

Begin cluster verification...

Error: Volume group /dev/vg00 is configured differently on node pfs1 than on node pfs1b
Error: Volume group /dev/vg00 on node pfs1b does not appear to have a physical volume corresponding to /dev/dsk/c1t6d0 on node pfs1 (617329303966546317).
Error: Volume group /dev/vg00 on node pfs1b does not appear to have a physical volume corresponding to /dev/dsk/c2t6d0 on node pfs1 (617329303990878847).
Error: Volume group /dev/vg00 is configured differently on node pfs1b than on node pfs1
Error: Volume group /dev/vg00 on node pfs1 does not appear to have a physical volume corresponding to /dev/dsk/c1t6d0 on node pfs1b (608329393992142354).
Error: Volume group /dev/vg00 on node pfs1 does not appear to have a physical volume corresponding to /dev/dsk/c2t6d0 on node pfs1b (608329393992142352).
Warning: The volume group /dev/vg00 is activated on more than one node:
pfs1
pfs1b
Warning: Volume groups should not be activated on more than one node.
Use vgchange(1m) to de-activate a volume group on a node.
cmcheckconf : Unable to reconcile configuration file /etc/cmcluster/pfs1_clust.conf
with discovered configuration information.


I've never seen this. Help.

[Carsten Krege]

SG will think that a VG is on the shared bus if it can see the identical volume group id (VGID) on two different nodes. This is usually a good assumption, if the machines were installed in a supported way. ;)

What we often see (and do not support) is that customers clone their disks with tools like dd(1m) or with copyutil (in the offline diagnostic). Also cloned disks created by hardware mirroring (e.g. EMC BCVs) are often seen. The common scenario is then that the LVM header is the same on the boot disks, thus confusing SG as the VGID is now the same.

The only way to get rid of this, is to change the VGID of one of the VGs by doing the following:

- boot one machine to maintenance mode (hpux -lm)
- vgchgid -f /dev/dsk/ /dev/dsk/ ..
(include all disks of vg00 in this command, -f (force) option is undocumented)
- vgexport vg00
- mkdir /dev/vg00
- mknod /dev/vg00/group c 64 0x0
- vgimport /dev/vg00 /dev/dsk/... /dev/dsk/..
- reboot

If the above scenario does NOT describe your situation, you may want to check the following:

- Has the /etc/lvmtab been copied from one node to the other?
In this case you should also follow the same procedure above (vgexport/vgimport in maintenance mode), but NOT run vgchgid (since only the lvmtab contains the wrong VGID, but the LVM header of the disks is ok)

- For specific diskarrays (e.g. Nike diskarray) SG checks the serial number. The serial numbers must be different. This can be checked by your h/w support. If the serial numbers of two diskarrays are the same, it can have a similar effect.

Carsten

[Denver Osborn]

I've seen this occur when a make_recovery tape is used to "clone" identical systems that are then placed into an MC/SG Cluster. One of the boxes had to be reinstalled.

[Mark van Hassel]

Remove /dev/vg00 from your cluster configuration file and then run the cmcheckconf or cmapplyconf again.

[Laurent Paumier]

Has vg00 been configured as a cluster managed vg ? try a "vgchange -c n /dev/vg00" and check again ?

[Daryl Much]

for the record, if running 11i, Carsten's suggestion is almost correct. I did it and had an unbootable system, but I could enter lvm maintenance mode (hpux -lm). The procedure I used was:

- boot to lvm maintenance mode (hpux -lm)
- vgexport vg00 (do this 1st or vgchgid fails!)
- vgchgid -f /dev/rdsk/
(list all disks in vg in one command)
- mkdir/mknod vg00 files
- vgimport vg00

then, you need to do the lvlnboot commands on the /, /stand, and swap lv's:

- lvlnboot -b /dev/vg00/lvol# (/stand lv)
- lvlnboot -r /dev/vg00/lvol# (/ lv)
- lvlnboot -s /dev/vg00/lvol# (swap lv)
- lvlnboot -d /dev/vg00/lvol# (swap lv)
- lvlnboot -R /dev/vg00
- lvlnboot -v /dev/vg00
- mount /dev/vg00/lvol# /usr (mount /usr)
- lifcp /dev/rdsk/c1t6d0:AUTO - (confirm 'hpux' is listed)

Then reboot. worked for me - YMMV.

Regards,

Chuck Davis