cmgetconf error

[Neeraj Bajpai]

Hi All,
I am having error while running cmgetconf command, I checked the forum but still can not resove this problem.
I need to change the lock disk of my two node cluster, for that I am running bellow command
# cmgetconf -v -c cl.conf
but it gives this error
********************************************
Gathering configuration information .... Done
Warning: The disk at /dev/dsk/c0t2d0 on node dbsrv does not have an ID,
or
a disk label. ---------> this is a cdrom device
Warning: The disk at /dev/dsk/c0t2d0 on node appsrv does not have an ID,
or
a disk label.
Warning: Disks which do not have IDs cannot be included in the topology
description.
Use pvcreate(1M) to initialize a disk for LVM or,
use vxdiskadm(1M) to initialize a disk for VxVM.
Warning: Volume group /dev/vg00 is configured differently on node dbsrv
than on node appsrv
Error: Volume group /dev/vg00 on node appsrv does not appear to have a
physical volume corresponding to /dev/dsk/c2t6d0 on node dbs.
Warning: Volume group /dev/vg00 is configured differently on node appsrv
than on node dbsrv
Error: Volume group /dev/vg00 on node dbsrv does not appear to have a
physical volume corresponding to /dev/dsk/c2t6d0 on node apps.
Warning: The volume group /dev/vg00 is activated on more than one node:
dbsrv
appsrv
Warning: Volume groups should not be activated on more than one node.
Use vgchange to de-activate a volume group on a node.
**********************************************
and after that it did not create any cluster configuration file.

I also tried this
# cmquerycl -n node1 -n node2 -C cl.conf

but no luck, it gives the same error with
*******************************************
Failed to gather configuration information.
*******************************************
thanks in advance

[Carsten Krege]

Most likely vg00 has the same volume group id on both systems. Either you have copied /etc/lvmtab from one node to the other or you have installed the machine using "copyutil" or "dd" or similar disk level copy utilities.

It appears that SG thinks the vg00's are on the shared bus because it finds the same vg id on both nodes for it. For local volume groups this should never be the case.

Without going into too much details, because the procedure is somewhat complex and depends on the specific setup you have, I suggest you either change the vgid of the vg00 on one node (vgchgid on vg00 in maintenance mode and vgexport/vgimport of vg00) or make /etc/lvmtab correct (vgexport/vgimport vg00 in maintenance mode).

Carsten

[Neeraj Bajpai]

Thanks Carsten,

As I understand that in LVM maintenance mode
I should do the fallowing things
Step 1.
# vgchid /dev/rdsk/c1t6d0 /dev/rdsk/c2t6d0
Step 2.
take the note of the /dev/vg00/group file major and minor numbers
Step 3.
# vgexport -p -m vg00.map /dev/vg00 and then
run the "vgexport" to remove the /etc/lvmtab entry.
Step 4.
# mkdir /dev/vg00
# mknod /dev/vg00/group c 64 0x000000
Step 5.
and then simply import the group by
# vgimport -p -m

Pl. correct me if I am fallowing wrong step.

and thanks for your precious advice â

[Carsten Krege]

This sounds very good, I only see some minor errors:

In LVM maintenance mode you should do the fallowing things
Step 1.
# vgchgid /dev/rdsk/c1t6d0 /dev/rdsk/c2t6d0

(make sure you specify all physical disks in vg00 , but not the alternate links)

Note: this might require the -f (force) option (undocumented).

Step 2.
take the note of the /dev/vg00/group file major and minor numbers

Step 3.
# vgexport -p -m vg00.map /dev/vg00 and then
run the "vgexport" to remove the /etc/lvmtab entry.

Step 4.
# mkdir /dev/vg00
# mknod /dev/vg00/group c 64 0x000000

Step 5.
and then simply import the group by
# vgimport -m /dev/vg00 /dev/dsk/c1t6d0 /dev/dsk/c2t6d0

(drop the -p option and specify the disk devices)

However, before you start you should verify that the VG ids are indeed the same on both nodes. You can do this by executing

vgexport -p -s -m /tmp/mapfile /dev/vg00

on both nodes and comparing if the first line in /tmp/mapfile is on both nodes the same.

[Neeraj Bajpai]

Excelent information, I checked both the system and found the same VGID I will fallow the step as you corrected.

Once again thanks for your time and help.

[Roberto Volsa]

Hi,
any problem in deactivating a busy volume group? Vg00 will be busy since the boot of the node.

BR

[Neeraj Bajpai]

Rob,

I have not tried yet because my system is in production. I will try later..

Do you think there will be a problem in deactivating the /dev/vg00 volume group in maintenance mode.

b'cos i have not tried this before...
regards

[Kent Ostby]

All --

When you boot into -lm mode, none of the disks will be active so this will not be a problem.

This is specifically the reason to boot in -lm.

Best regards,

Kent M. Ostby

[Neeraj Bajpai]

Hi carsten ,

I tried the procedure as you advised in your previous discussion for changing the VG ID for root disk in cluster enviornment.. I did the fallowing thing but could not able to do it it gives this error ...

# vgchgid /dev/rdsk/c1t6d0 /dev/rdsk/c2t6d0
vgchgid: disk "/dev/rdsk/c1t6d0" is already in vg "/dev/vg00"

I did this in lvm maintenance mode, What does it means ..

Will you please tell me....
Regards

[Stephen Doud]

Perhaps this document will help?


http://ITRC.HP.COM document ID: UXSGKBRC00007573
TITLE: Cloned vg00 and ServiceGuard configuration failures


CONFIGURATION
HPUX versions 10.20 and 11.x with MC/Serviceguard

PROBLEM

The cmquerycl command fails when more than one node is specified as in:
# cmquerycl -v -n node1 -n node2

/var/adm/syslog/syslog.log logs only these messages:

cmclconfd[20688]: LVM query message
cmclconfd[20688]: get_tcp_messages:receive != 0 fd=0

cmquerycl completes without errors when only a single node is specified.
What can be the cause of this condition?

NOTE: In 10.X version clusters, the following error messages may be seen:

Cannot access cluster info: The physical volume with name
/dev/dsk/c2t6d0 on node adac2 cannot be found on node adac1

Cannot access cluster info: The physical volume with name
/dev/dsk/c2t6d0 on node adac1 cannot be found on node adac2

RESOLUTION
The error messages infer that cmquerycl detected a faulty LVM condition.
Although not explicitly detailed, in this case the cmquerycl command
may be failing because vg00 was cloned from another system in the cluster and
vg00's between servers do not look alike.

Details:
When a volume group is created, it is given a unique VGID - a merger of the
servers' machine ID (uname -i) and the timestamp of the VG creation date. To
save time, administrators may have used dd or copyutil to clone
vg00 onto another servers' disks. Unfortunately, this also copies the same
VGID to the new server.

MC/ServiceGuard utilizes LVM structures such as Volume Group ID (VGID) and
Physical Volume ID (PVID) to determine which VGs are shared (common to both
servers). If each servers' vg00s' PVID and VGID are the same, cmquerycl
(ServiceGuard) considers them to be the same VG. Subsequent alterations of
vg00 LVM structures (such as adding a disk) are interpretted by cmquerycl as
unresolvable LVM differences between servers - terminating the command.

To determine if vg00 was cloned from another server in the cluster, compare
the VGID of vg00 on each server in the cluster. Perform these commands on
each node:

vgexport -ps -m /tmp/map.vg00 vg00
The -p option will allow the command to create a 'map' file and will not
export vg00!

grep VGID map.vg00
VGID 5930a1673819dd35 <<--- sample output

Compare the vg00 VGIDs. If any are not unique, those disks were cloned in an
"unsupported" manner. The "supported" resolution is to reinstall the cloned
boot disks. The recommended way is to create a make_recovery restore
tape and install from it. A make_recovery tape reinstalls vg00 with the
custom configuration of the server. In doing so, it creates a new VGID for
vg00 at the time of the restore. See also KBRC00000779 -
Title: Problems using make_recovery to clone non-identical systems

--------------------------------------------------------------------------------
NOTES:
Though the following procedure has been used to correct the problem, HP has
not officially tested the procedure and therefore does not "support" its use
and will not warrant the remedy. Use at your own risk.

1) Before proceeding, insure the s700_800 LVM commands cumulative
baseline patch below or a successor is loaded on the server:
10.20: PHCO_14990
11.00: PHCO_27369
11.11: PHCO_27913

2) After loading patches for vgchgid, record the boot disk path(s) (cXtYdZ)
and backup the lvmtab file.

strings /etc/lvmtab (write down the disk path(s) for vg00)
cp /etc/lvmtab /etc/lvmtab.orig
shutdown -ry 0

3) Boot the server in LVM Maintenance mode. Interrupt the autoboot mechanism
to get to the ISL prompt and type this:

ISL> hpux -lm

4) Since the old VGID in /etc/lvmtab is not wanted, export vg00:
vgexport -v -m vg00.map /dev/vg00

5) Change vg00's VGID:
vgchgid -f /dev/rdsk/(raw_disk) /dev/rdsk/(raw_disk) ...

This vgchgid command MUST be run on ALL disks in a given volume group in
one command. Use the backslash "\" character to escape line feeds if the
command is longer than 255 characters.

6) RE-vgimport vg00:
mkdir /dev/vg00
mknod /dev/vg00/group c 64 0x000000
vgimport -v -m vg00.map /dev/vg00 /dev/dsk/(disk) ...

7) Update the LVM boot structures:
vgchange -a y vg00
BOOT: "lvlnboot -b /dev/vg00/lvol1"
ROOT: "lvlnboot -r /dev/vg00/lvol3"
Or lvol1 if their is NOT a seperate lvol for /stand

SWAP: "lvlnboot -s /dev/vg00/lvol2"
DUMP: "lvlnboot -d /dev/vg00/lvol2"

Enter "lvlnboot -R /dev/vg00"

Display the BOOT LIF to confirm it is all there:
"lvlnboot -v /dev/vg00"
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c1t6d0 (0/0/2/0.6.0) -- Boot Disk
/dev/dsk/c3t0d0 (0/4/0/0.0.0)
/dev/dsk/c5t0d0 (0/12/0/0.8.0.255.0.0.0)
Boot: lvol1 on: /dev/dsk/c1t6d0
/dev/dsk/c3t0d0
Root: lvol3 on: /dev/dsk/c1t6d0
/dev/dsk/c3t0d0
Swap: lvol2 on: /dev/dsk/c1t6d0
/dev/dsk/c3t0d0
Dump: lvol2 on: /dev/dsk/c1t6d0, 0

8) Update the LVM boot command if needed:
$ mount /dev/vg00/lvolN /usr
- select the proper lvol from /etc/fstab (lvol7?)

9) Verify the boot command is simply "hpux":
$ lifcp /dev/rdsk/c1t6d0:AUTO - note the 'minus' sign
hpux

If it needs to be changed, use this example syntax:
$ mkboot -a "hpux" /dev/rdsk/c1t6d0

10) Reboot. The VGID is now unique for vg00.

NOTE: See also A5092467

### END ###

Search the http://itrc.hp.com technical knowledge database (login required) for
more ServiceGuard documents using the prefixes of UXSG* or UMCSG*

[Carsten Krege]

As I mentioned in my 2nd reply: YOu might need the "-f" (force) option for vgchgid.

See also Stephen's comment.

Carsten

[Neeraj Bajpai]

I tried this too ....
vgchgid -f /dev/rdsk/c1t6d0 /dev/rdsk/c2t6d0
vgchgid: disk "/dev/rdsk/c1t6d0" is already in vg "/dev/vg00".

but the same error mesg.

I will try stephens method too and let you know..

Thanks for your time.