Help with Cluster configuration

[arkie]

Hi All,

I am trying to create a SG cluster as a learning experience. Pls see the below

# vgchange -c y vg01
vgchange: Cluster Services not available or unreachable on this system, for volume group "/dev/vg01".

Am i missing some critical software component?

Pls help

[Horia Chirculescu · Accepted solution]

>Cluster Services not available

Did you checked if the cluster daemon is running? Did you started the cluster?

Horia.

[arkie]

Hi Horia,

`ps -ef|grep -i cmcld` is indicating that the cluster services are not running. How do i start the cluster daemon?

[rariasn]

Hi,

The high availability software must be running;otherwise, the volume group is not marked.
Needs to be done on one node only.

man vgchange

rgs,

[S.N.S]

Hi Arkie,

cmcld is HP SG's main daemon.

To start the whole cluster:

# cmruncl

(Assuming that you have never run the command on any of the nodes of the Cluster)
Let us know if you get any error doing this.

HTH
SNS

[Horia Chirculescu]

>How do i start the cluster daemon?

The daemon is started when the cluster is started

If you need to run the cluster daemon on only one node then:

cmruncl -n


Check also /sbin/init.d/cmcluster

Horia.

[Matti_Kurkela]

If you haven't created a cluster configuration yet, let me give you a small bit of advice.

Please, READ THE MANUAL FIRST. Setting up a cluster is a complex task, and the "Managing Serviceguard" manual explains all the necessary details very clearly.

The "Managing Serviceguard" manual comes with every Serviceguard license as an actual physical book. It is also available in PDF and HTML forms in the High Availability section of the docs.hp.com website:

http://docs.hp.com/en/ha.html#Serviceguard

The actual cluster set-up begins with:
1.) install Serviceguard software on all the nodes you're going to use

2.) write a /etc/cmcluster/cmclnodelist file to authorize Serviceguard access while the cluster configuration is being built.

3.) run the "cmquerycl" command with the appropriate parameters to create your cluster configuration template

4.) modify the template created by cmquerycl to match your requirements (YOU WILL NEED THE MANUAL TO DO THIS STEP CORRECTLY.)

5.) run the "cmcheckconf -C " command to verify that the modified configuration is good

6.) run the "cmapplyconf -C " command to actually implement the configuration

7.) run "cmruncl" to start the cluster infrastructure

Only after all these steps are successfully completed, the "vgchange -c y" command can be used.

MK

[John Bigg]

If all the steps suggested by Matti have been done correctly, then the vgchange -c y will not even be needed since this equivalent of this operation will have performed by the cmapplyconf!

I am constantly amazed at the number of people who attempt to setup a cluster without having the knowledge to do so. You will only achieve high availability if you have setup a sensible configuration and this will only be done if you understand what you are doing rather than following a checklist written by someone else.

[S.N.S]

Hi,

I need to apologize - I too had to advice You on what Matti rightly said.
And with HP SG - you need to be careful and alert... Read thru the docs very well - and kindly excuse this extra post.

Danke,
SNS

[arkie]

Hi All,

Thanks for all these valuable inputs, but I am rather new and hence, seek more of your guidance at various stages. I am trying to configure a test cluster with 2 nodes. At this stage, I am getting the below reply to my `cmquerycl -v -C ` :-

# cmcheckconf -v -C /etc/cmcluster/cseb.ascii

Checking cluster file: /etc/cmcluster/cseb.ascii
Note : a NODE_TIMEOUT value of 2000000 was found in line 144. For a
significant portion of installations, a higher setting is more appropriate.
Refer to the comments in the cluster configuration ascii file or Serviceguard
manual for more information on this parameter.
Checking nodes ... Done
Checking existing configuration ... Done
Warning: Unable to get configuration for cluster testcluster.
Gathering configuration information ... Done
Gathering configuration information ..
Gathering storage information ..
Found 4 devices on node drapp1
Found 4 devices on node drapp2
Analysis of 8 devices should take approximately 1 seconds
0%----10%----20%----30%----40%----50%----60%----70%----80%----90%----100%
Found 3 volume groups on node drapp1
Found 3 volume groups on node drapp2
Analysis of 6 volume groups should take approximately 1 seconds
0%----10%----20%----30%----40%----50%----60%----70%----80%----90%----100%
.....
Gathering Network Configuration ......... Done
Error: First cluster lock volume group /dev/vg03 cannot be found in the cluster.
Error: First cluster lock volume group /dev/vg03 needs to be designated as a cluster aware volume group.
cmcheckconf : Unable to verify cluster file: /etc/cmcluster/cseb.ascii.

Pls provide your valuable inputs on where i am going wrong. FYI i have 2 VGs and propose to use VG03 as my LOCK VG

[arkie]

Would also like to add the following:-

`cmcheckconf -v -C /etc/cmcluster/cseb.ascii` output contained the below message:-

Error: First cluster lock volume group /dev/vg03 needs to be designated as a cluster aware volume group.

So, after `vgchange -a n vg02` on Node 1, tried to `vgchange -c y vg02` on Node 2, but the result is:-

# vgchange -c y vg02
vgchange: Cluster Services not available or unreachable on this system, for volume group "/dev/vg02".

My confusion is - how can the cluster service be available at this stage?

[rariasn]

Hi,
Sample cluster configuration.

1.- configure "cluster lock"

FIRST_CLUSTER_LOCK_VG /dev/vg02

NODE_NAME node1
NETWORK_INTERFACE lan1
HEARTBEAT_IP 172.23.5.1
NETWORK_INTERFACE lan0
FIRST_CLUSTER_LOCK_PV /dev/dsk/cxtydz

NODE_NAME node2
NETWORK_INTERFACE lan1
HEARTBEAT_IP 172.23.5.2
NETWORK_INTERFACE lan0
FIRST_CLUSTER_LOCK_PV /dev/dsk/cxtydz

2.- Verify configuration

# cmcheckconf ....

3.- Apply configuration

# cmapplyconf ....

4.- run a high availability cluster

5.- Modify VG

# vgchange -a e vg02
# vgchange -a e vgxx
# rmcuncl

http://www.docs.hp.com/en/B3936-90079/B3936-90079.pdf

rgs.

[arkie]

Hi Rariasn,

I propose to use vg03 and /dev/dsk/c5t0d2 as the LOCK VG and LOCK PV respectively. Can your pls share the required steps from each of the two (2) nodes for the lock device.

Thanks in advance

[Matti_Kurkela]

Your vg03 volume group must already exist (as a regular non-cluster VG) and be imported to both nodes at the time you run your cmcheckconf/cmapplyconf.

Bottom line: you *cannot* use "vgchange -c y" until your cluster is running. That command is useful only when adding new packages to an existing, running cluster. So don't get hung up on it now.

As you don't have a cluster yet, you cannot have any cluster-aware VGs. But the lock VG needs to be cluster-aware.

The solution is: when you run cmapplyconf to create a cluster, the cmapplyconf procedure will simultaneously switch the VGs listed in the cluster ASCII file to cluster mode.

To make vg03 switch to cluster mode at the cluster creation time (which is a requirement for using it as a cluster lock VG), you should add:

VOLUME_GROUP /dev/vg03

to the very end of your /etc/cmcluster/cseb.ascii file.

The appropriate FIRST_CLUSTER_LOCK_VG and FIRST_CLUSTER_LOCK_PV lines should also exist and be uncommented in the cseb.ascii file.

If you're planning to use vg02 and/or other VGs as cluster packages, you have the option of adding VOLUME_GROUP lines for them in the cseb.ascii file too. If you do this, you don't have to use "vgchange -c y" on those VGs later.

In a nutshell:
there are two methods to mark a VG as cluster-aware.

1.) specify it in a cluster ASCII file using a VOLUME_GROUP line, then apply the new
cluster configuration

2.) use "vgchange -c y"

When the cluster is down or not yet set up, only method 1 is available. Once your cluster is up and running, both methods will be available.

MK

[rariasn]

Hi,

A lock disk may be used for clusters up to and including four nodes in size. The cluster lock disk is a special area on an LVM disk located in a volume group that is shareable by all nodes in the cluster. When a node obtains the cluster lock, this area is marked so that other nodes will
recognize the lock as â taken.â

The lock disk is not dedicated for use as the cluster lock; the disk can be employed as part of a normal volume group with user data on it. The cluster lock volume group and physical volume names are identified in
the cluster configuration file.


If vg03 is sharable:

- node1:

# vgdisplay -v vg03|grep "PV Name"|tail -1|awk '{print $3}'

- node2:


# vgdisplay -v vg03|grep "PV Name"|tail -1|awk '{print $3}'

Use this information to configurate "cluster_lock"

rgs,

[arkie]

In both nodes:-

# vgdisplay -v vg01|grep "PV Name"|tail -1|awk '{print $3}'
/dev/dsk/c5t0d2

The output of cmcheckconf is saying:-

# cmcheckconf -v -C /etc/cmcluster/testcluster.config

Checking cluster file: /etc/cmcluster/testcluster.config
Note : a NODE_TIMEOUT value of 2000000 was found in line 141. For a
significant portion of installations, a higher setting is more appropriate.
Refer to the comments in the cluster configuration ascii file or Serviceguard
manual for more information on this parameter.
Checking nodes ... Done
Checking existing configuration ... Done
Warning: Unable to get configuration for cluster testcluster.
Gathering configuration information ... Done
Gathering configuration information ..
Gathering storage information ..
Found 3 devices on node drapp1
Found 3 devices on node drapp2
Analysis of 6 devices should take approximately 1 seconds
0%----10%----20%----30%----40%----50%----60%----70%----80%----90%----100%
Found 2 volume groups on node drapp1
Found 2 volume groups on node drapp2
Analysis of 4 volume groups should take approximately 1 seconds
0%----10%----20%----30%----40%----50%----60%----70%----80%----90%----100%
.....
Gathering Network Configuration ......... Done
Error: First cluster lock volume group /dev/vg01 needs to be designated as a cluster aware volume group.
cmcheckconf : Unable to verify cluster file: /etc/cmcluster/testcluster.config.

[rariasn]

Hi,

http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1272458230261+28353475&threadId=205207

try adding the line at the bottom of the ascii file:

VOLUME_GROUP /dev/vg01

You do not seem to have this entry, and for a cluster lock vg it must be defined in the ascii file

rgs,

[arkie]

Hi Rariasn,

After adding the entry in the config file, the output of `cmquerycl` has now changed to:-

Error: First cluster lock volume group "/dev/vg01" is not available on all nodes. Choose a volume group that is available on all nodes.
Error: Unable to find find first cluster lock volume group "/dev/vg01" on node "drapp2" but found it in cluster "testcluster".

I have checked and found vg01 is available in both both nodes

[rariasn]

Hi,

# cat /etc/cmcluster/testcluster.config

rgs,

[arkie]

Hi Rariasn,

Can you pls help review my `testcluster.config` file attached herewith

[S.N.S]

Arkie,

I did check your file;I have NOT found this entry:
VOLUME_GROUP /dev/vg01

Can you confirm this if you have made such an entry before you go further?

Am attaching a working cluster config file - pls check with it.

Let Us know what happens

HTH
SNS

[arkie]

Hi S.N.S.

I have added the entry `VOLUME_GROUP /dev/vg01` in /etc/cmcluster/testcluster.config, and tried `cmcheckconf`, but result was the same...particularly the below two error output:-

Error: First cluster lock volume group "/dev/vg01" is not available on all nodes. Choose a volume group that is available on all nodes.
Error: Unable to find find first cluster lock volume group "/dev/vg01" on node "drapp2" but found it in cluster "testcluster".
(Entire output attached in file)

So, I tried from scratch again:-

On Node 1 (drapp1):-

pvcreate -f /dev/rdsk/c5t0d2
mkdir /dev/vg01
mknod /dev/vg01/group c 64 0x010000
vgcreate /dev/vg01 /dev/dsk/c5t0d2
vgexport -p -v -s -m /tmp/vg01.map vg01
rcp -p /tmp/vg01.map drapp2:/tmp/

On Node 2 (drapp2):-

mkdir /dev/vg01
mknod /dev/vg01/group c 64 0x010000
vgimport -s -m /tmp/vg01.map /dev/vg01

But, I am getting the following output:-

# vgimport -s -m /tmp/vg01.map /dev/vg01
Couldn't access the list of physical volumes for volume group "/dev/vg01".

Pls let me know where i am going wrong.

[arkie]

Hi,

Would like to add the following:-

`vgimport -v -m /tmp/vg01.map /dev/vg01 /dev/dsk/c5t0d1` instead of `vgimport -v -s -m /tmp/vg01.map /dev/vg01` has helped.

Now I am getting no errors in the `cmcheckconf` output:-

But, I am getting a different error in the `cmapplyconf` outout:-

# cmapplyconf -v -C testcluster.config

Checking cluster file: testcluster.config
Note : a NODE_TIMEOUT value of 2000000 was found in line 127. For a
significant portion of installations, a higher setting is more appropriate.
Refer to the comments in the cluster configuration ascii file or Serviceguard
manual for more information on this parameter.
Checking nodes ... Done
Checking existing configuration ... Done
Warning: Unable to get configuration for cluster testcluster.
Gathering configuration information ... Done
Gathering configuration information ..
Gathering storage information ..
Found 4 devices on node drapp1
Found 4 devices on node drapp2
Analysis of 8 devices should take approximately 1 seconds
0%----10%----20%----30%----40%----50%----60%----70%----80%----90%----100%
Found 3 volume groups on node drapp1
Found 3 volume groups on node drapp2
Analysis of 6 volume groups should take approximately 1 seconds
0%----10%----20%----30%----40%----50%----60%----70%----80%----90%----100%
.....
Gathering Network Configuration ......... Done
Checking for inconsistencies .. Done
Maximum configured packages parameter is 1.
Configuring 0 package(s).
1 package(s) can be added to this cluster.
200 access policies can be added to this cluster.
Marking/unmarking volume groups for use in the cluster
0%Creating the cluster configuration for cluster testcluster.
Adding node drapp1 to cluster testcluster.
Adding node drapp2 to cluster testcluster.
Protocol failure talking with cmclconfd on drapp1 (1) : Cross-device link
Error: Unable to initialize Volume Group /dev/vg01 on node drapp1.
Check the syslog file on that node for more information.
cmapplyconf : Unable to apply the configuration

############################################
Will be very thankful for any help

[S-M-S]

Hi Arkie,
I hope you are running the command from node drapp1

Check whether vglock is activated on the same node from you are running the command

[rariasn]

Hi arkie,

In node "drapp1":

# vgchange -a y vg01
# vgdisplay -v vg01
# vgchange -a n vg01

In node "drapp2":

# vgchange -a y vg01
# vgdisplay -v vg01

rgs,