Changing the cluster lock volume group and disk

[EJ Stremler]

We used to have our cluster lock Volume Group on SAN and it got inadvertently removed.. I know I need to shut the cluster down to initialize a new cluster lock disk. Can someone check my sequence of commands to do this? Thanks, Ed....
1. vgexport oldclvg
2. create the newclvg with the new cluster lock disk.
3. vgimport newclvg on adoptive node
3. Edit the cluster.ascii file with the newclvg and new disk device.
4. cmhaltpkg "packages"
5. cmhaltcl
6. vgchange -a y /dev/newclvg from one of the nodes.
7. cmapplyconf -k -v -C cluster.ascii
8. vgchange -a n /dev/newclvg
9. cmruncl
10. cmrunpkg "packages"

[Sundar_7 · Accepted solution]

Ed,

In this case, my approach would be to let the cluster itself suggest a lock disk for me.

# cd /etc/cmcluster
# cmquerycl -v -C new.cluster -n node1 -n node2 -n ...

Edit the new.cluster to make sure it detected soem other VG as the cluster VG and PV

# cmcheckconf -v -C new.cluster
# cmhaltcl -f
# cmapplyconf -v -C new.cluster
# cmruncl -v

Sundar

[Mridul Shrivastava]

If any of the FIRST_CLUSTER_LOCK_* values must be changed, follow this
procedure:

PROCEDURE:
1) Locate the cluster ASCII file. If it's whereabouts is unknown, recreate
it on one of the nodes in the cluster:

# cd /etc/cmcluster
# cmgetconf cluster.ascii

2) Insure the cluster.ascii reflects the current hardware and network
configuration:

# cmcheckconf -C cluster.ascii

If this command fails, the cluster.ascii may be out-dated. Either correct
the problem, or dissolve (cmdeleteconf -f) the current cluster and
create a fresh cluster.ascii.
Example:
# cmhaltcl -f
# cmdeleteconf -f
# cmquerycl -C cluster.ascii -n -n ...

3) Locate the FIRST_CLUSTER_LOCK_VG and FIRST_CLUSTER_LOCK_PV references in
the cluster.ascii file.

4) If the FIRST_CLUSTER_LOCK_VG reference will be changed, and it is a new
volume group for the cluster, insure that each node can see the same set
of disks for that volume group. See /etc/lvmtab on each node.
Then add the volume group name to the cluster.ascii file. Example:

VOLUME_GROUP /dev/vg03

j) Update the FIRST_CLUSTER_LOCK_VG reference and/or FIRST_CLUSTER_LOCK_PV
references in the file.

6) When a maintenance window arrives, halt the cluster:

# cmhaltcl -f

7) Activate the cluster lock VG. NOTE: If the lock VG is already owned
by the cluster, it must first be "declustered" before it can be
activated. Example:

# vgchange -c n vg03
# vgchange -a y vg03


8) Use the cmapplyconf command now. This command performs both a
cmcheckconf validation of the cluster.ascii file, and then updates and
distributes the cluster binary file:

# cmapplyconf -f -C cluster.ascii
Begin cluster verification...
Modifying node in cluster
Modifying node in cluster

It should not be necessary to delete the cluster (cmdeleteconf) prior to changing the cluster lock volume group or disk.

[David de Beer]

Greetings, we had a similar problem a few weeks ago. The cmapplyconf was not enough to restore the needed cluster structures on the lock disk - the cluster physically (binary? :) had to be Deleted (cmdeleteconf -v -c cluster_name), and cmapplyconf done again from scratch. (I do not know what version it was, sorry).

A much cleaner approach would be to install a Quorum Server for your network. This works really wonderful. It's so small, that if you had a problem with the host that it's configured on, you could easily install it on another host and configure the IP (if it's in the same subnet). The Quorum server is free (as you probably know) and you can download it from software.hp.com

There is a catch tho, you cannot change the IP address of the Quorum server without taking down your clusters (You can move it to another machine, just in the same subnet - online).

You could then use the Quorum Server for your other clusters aswell.

Kind Regards,
David.

[Kevin Wright]

re-run cmquerycl to re-create a new ascii file, then merge them together.

to obtain current asci, run cmgetconf.

[Stephen Doud]

Your steps look correct Ed :)

[EJ Stremler]

Thanks all for your help, it is all valuable information.. And thanks Stephen for your over the phone support and also solving the permissions problem with the cluster...