Redhat cluster is not working properly

[senthil_kumar_1]

Could any one please help me on this...

[Matti_Kurkela]

When changing the configuration, you are supposed to increase the config_version value on the first line each time. Otherwise your changes may not be recognized by the running cluster.

The configuration you copy&pasted indicates you did not update the config_version. Before your change:

<cluster alias="clu" config_version="14" name="clu">

After your configuration change:

<cluster alias="clu" config_version="14" name="clu">

Increase the config_version and try again.

The proper procedure for modifying the configuration while the cluster is running depends on the RHEL version.

With RHEL 5, you should not modify /etc/cluster/cluster.conf directly: instead, you should make a copy of it, make changes to the copy (remember to increase config_version!), and then use "ccs_tool update <modified copy of cluster.conf>" to make the changes effective. The ccs_tool will automatically verify the configuration file, propagate it to all the cluster nodes, and then all the cluster nodes can update their configuration files in a synchronized fashion.

With RHEL 6, you can apparently edit /etc/cluster/cluster.conf while the cluster is running, but the changes will take effect only after you run "cman_tool version -r". Before you do that, you should run "ccs_config_validate" to verify the configuration syntax is OK.

[senthil_kumar_1]

Hi All,

Now file system is working fine with following configuration.

# more /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster alias="BIOSDB" config_version="15" name="BIOSDB">
        <quorumd interval="1" label="osdb_qdisk" min_score="1" tko="10" votes="1">
                <heuristic interval="2" program="10.250.0.1" score="1"/>
        </quorumd>
        <fence_daemon post_fail_delay="0" post_join_delay="20"/>
        <clusternodes>
                <clusternode name="emdlagpbw01.emdna.emdiesels.com" nodeid="1" votes="1">
                        <fence>
                                <method name="1">
                                        <device name="EMDLAGPBW01R"/>
                                </method>
                        </fence>
                </clusternode>
                <clusternode name="emdlagpbw02.emdna.emdiesels.com" nodeid="2" votes="1">
                        <fence>
                                <method name="1">
                                        <device name="EMDLAGPBW02R"/>
                                </method>
                        </fence>
                </clusternode>
        </clusternodes>
        <cman expected_votes="1" two_node="1" broadcast="yes"/>
        <fencedevices>
                <fencedevice agent="fence_ipmilan" power_wait="10" lanplus="1" ipaddr="10.254.1.113" login="tcs" name="EMDLAGPBW01R" passwd="tCs12345"/>
                <fencedevice agent="fence_ipmilan" power_wait="10" lanplus="1" ipaddr="10.254.1.143" login="tcs" name="EMDLAGPBW02R" passwd="tCs12345"/>
        </fencedevices>
        <rm>
                <failoverdomains>
                        <failoverdomain name="EMDLAGPBWCL1" ordered="1" restricted="1">
                                <failoverdomainnode name="emdlagpbw01.emdna.emdiesels.com" priority="1"/>
                                <failoverdomainnode name="emdlagpbw02.emdna.emdiesels.com" priority="2"/>
                        </failoverdomain>
                        <failoverdomain name="EMDLAGPBWCL2" ordered="1" restricted="1">
                                <failoverdomainnode name="emdlagpbw01.emdna.emdiesels.com" priority="2"/>
                                <failoverdomainnode name="emdlagpbw02.emdna.emdiesels.com" priority="1"/>
                        </failoverdomain>
                </failoverdomains>
                <resources>
                        <ip address="10.250.1.107/22" monitor_link="1"/>
                        <script file="/etc/init.d/httpd" name="httpd"/>
                        <lvm name="lvm1" vg_name="test_vg" lv_name="test_lvol1"/>
                        <fs device="/dev/test_vg/test_lvol1" force_fsck="0" force_unmount="1" fsid="33611" fstype="ext3" mountpoint="/test_node1_filesystem" n
ame="test_node1" options="" self_fence="0"/>
                        <ip address="10.250.1.108/22" monitor_link="1"/>
                        <script file="/etc/init.d/vsftpd" name="vsftpd"/>
                        <lvm name="lvm2" vg_name="test_new_vg" lv_name="test_new_vg_lvol1"/>
                        <fs device="/dev/test_new_vg/test_new_vg_lvol1" force_fsck="0" force_unmount="1" fsid="54001" fstype="ext3" mountpoint="/test_node2_fi
lesystem" name="test_node2" options="" self_fence="0"/>
                </resources>
                <service autostart="1" domain="EMDLAGPBWCL1" name="httpd" recovery="relocate">
                        <ip ref="10.250.1.107/22"/>
                        <script ref="httpd"/>
                        <lvm ref="lvm1"/>
                        <fs ref="test_node1"/>
                </service>
                <service autostart="1" domain="EMDLAGPBWCL2" name="vsftpd" recovery="relocate">
                        <ip ref="10.250.1.108/22"/>
                        <script ref="vsftpd"/>
                        <lvm ref="lvm2"/>
                        <fs ref="test_node2"/>
                </service>
        </rm>
</cluster>

Now I would like to configure Quorum Disk (Qdisk) with this configuration...

For that, I have configured disk "/dev/sda1" as Qdisk and it is visible in both nodes...

Now How can I configure Qdisk in above configuration file?

[senthil_kumar_1]

Could any one of you all help me to configure Qdisk for above configuration file...

[Matti_Kurkela]

 This line in your cluster configuration does not seem to be correct:

            <heuristic interval="2" program="10.250.0.1" score="1"/>

 The program= field must be a complete command, not just an IP address. You probably meant something like:

            <heuristic interval="2" program="ping -c1 -t1 10.250.0.1" score="1"/>

 When you fix this, remember again to increase the config_version value.

Otherwise your quorum disk configuration looks OK to me.

The next step is to prepare the quorum disk with "mkqdisk".

On one node, run:

mkqdisk -c /dev/sda1 -l osdb_qdisk

 On the other node, run this command to verify the node sees the newly-created quorum disk:

mkqdisk -L

The output should look like this:

# mkqdisk -L
mkqdisk v0.6.0
/dev/sda1:
        Magic:                eb7a62c2
        Label:                osdb_qdisk
        Created:              Thu Mar 18 15:29:49 2010
        Host:                 emdlagpbw01
        Kernel Sector Size:   512
        Recorded Sector Size: 512

Then you can start the quorum disk daemon on both nodes:

service qdiskd start
chkconfig qdiskd on

After starting the quorum disk daemon, wait a few minutes, then run "clustat". The quorum disk should appear in the cluster member listing, typically with a node ID 0 and with Status "Online, Quorum Disk".

[senthil_kumar_1]

Hi Matti,

Have configured Qdisk successfully...Thanks a lot...


Now I have some questions.

1)Can we see the data stored in Qdisk?

2)What is the default deadnode_timeout value for RHEL 5.7?

3)How to see deadnode_timeout set right now?

4)How to set deadnode_timeout value in RHEL 5.7?

[lickky]

hi Jimmy, 

I think you are right. I create a user for fence_ipmilan and execute the command:

fence_ipmilan -A password  -a XX.XXX.XXX.XX -l RHCS_USER -p 1qaz2wsx -o status -v -P, it turns out failed. after read your reply, i noticed that i didn't give it the administrator privilege but only a user privilege having right to reset server. then i grant my user as administrator, it succeed.

Thank you very much. It is really helpful for me.