topic Re: Red Hat Cluster suite 5 question in Operating System - Linux

Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Thu, 13 Mar 2008 08:41:15 GMT

Hi All,

Cluster package unable to failover when node 1 when press the power button directly. Even though has already upgrade the ilo firmware from 1.30 to 1.42.

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Thu, 13 Mar 2008 09:00:10 GMT

Is heartbeat is still been use in red hat cluster suite 5.
Appreciate anyone that reply and help.

thanks.

Re: Red Hat Cluster suite 5 question

Ivan Ferreira — Thu, 13 Mar 2008 11:53:44 GMT

Probably node 2 cannot fence node 1 properly. What fence device are you using?

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Thu, 13 Mar 2008 14:44:26 GMT

Hi,

I'm using hp ilo to fence both node. Both node 1 and node to fence to another server hp ilo.

thanks.

Re: Red Hat Cluster suite 5 question

Ivan Ferreira — Thu, 13 Mar 2008 14:55:10 GMT

What do you get in /var/log/messages, have you confirmed that fencing is working?

Re: Red Hat Cluster suite 5 question

Steven E. Protter — Thu, 13 Mar 2008 15:06:21 GMT

Shalom,

Test manually that you can connect to ilo and do a power reset on the server.

RH4 required a password be hard coded in the cluster.conf file. Use that password to manually test ilo.

This could be a simple problem of not having the correct authentication. It could also be a bad ilo card.

service fenced status

both nodes please.

iLo cards need to be on the same network and be able to talk to each other.

SEP

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Fri, 14 Mar 2008 00:10:33 GMT

Hi all,

How should my fence physical connection to be. As, Physical connection is that both node only use 2 network port to do bonding and i just use another server ilo port to fence both node. Is this workable. All the ipaddress has been configure in the same network/ segment.

Thanks.

Re: Red Hat Cluster suite 5 question

Ivan Ferreira — Fri, 14 Mar 2008 02:03:47 GMT

It does not matter as long as the ILO and your public network are in the same subnet. What about the messages? Is your ILO configuration correct?

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Fri, 14 Mar 2008 08:18:53 GMT

I have check the configuration few times. I'm sure i have configure properly. In the cluster log there is no error for the ilo. Besides cluster log is there another log for ilo?

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Mon, 17 Mar 2008 11:15:55 GMT

this cluster suite is making me go crazy. Today try to setup the fence device by using HP ilo where by using the both node ilo port. Someone has hardcode the ilo on node 2(ml370 g3)now everytime i press f8 it will prompt for username and password. did ask the owner on that machine but don't remember the username and password. Is there a way to reset the ilo.?

Re: Red Hat Cluster suite 5 question

Steven E. Protter — Mon, 17 Mar 2008 13:33:01 GMT

Shalom,

To reset ilo to factory defaults.

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=845909

SEP

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Fri, 21 Mar 2008 00:48:12 GMT

Hi all,

Is this normal when i configure hp-ilo on the same segment with the public ip. When press the power button on node 1, the node when down and few seconds later the node power up by itself and the service only failover to node 2.

Re: Red Hat Cluster suite 5 question

Ivan Ferreira — Fri, 21 Mar 2008 12:11:55 GMT

I think it is, as when you power down node 1, node 2 will try to fence it. By default, fence_ilo reboots the server, according to man fence_ilo, there is an option to power down the server instead of restarting.

And yes, it will failover to node 2, and unless you configure a failover group with priotiy, will stay there.

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Mon, 24 Mar 2008 01:25:01 GMT

Hi Ivan,

I have man the fence_ilo. It the descrition it say's that fence_ilo -0 is to set the option, But where should i set it, just open a teminal and set it or do i need to hard code it somewhere? e.g cluster config file / fence_ilo script.

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Mon, 24 Mar 2008 12:52:17 GMT

After changing the action from reboot to off in fence_ilo script, now when press the power button the service able to failover. If the power cable have been plug out then the service wont failover. is this the limitation for fence_ilo.?

Re: Red Hat Cluster suite 5 question

Ivan Ferreira — Mon, 24 Mar 2008 13:15:20 GMT

Yes, as the remaining node cannot confirm that the other was fenced. You can then configure other fence device, as brocade if you use sanswitch.

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Mon, 31 Mar 2008 05:58:01 GMT

Facing another problem here. Base on the /var/log/messages log show that after 30 mins the cluster service restart the application by itself an return error code 1. how do i debug this error message.

error log

Mar 28 07:25:15 BA-GW1 clurgmgrd: [4751]: script:MDCS_QUERY_SERVICE: status of /etc/init.d/queryService failed (returne
d 1)
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: status on script "MDCS_QUERY_SERVICE" returned 1 (generic error)
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Stopping service service:MDCS_QUERY_SERVICE
Mar 28 07:25:15 BA-GW1 clurgmgrd: [4751]: script:MDCS_GATEWAY_SERVICE: status of /etc/init.d/mdcs failed (returned 1)
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: status on script "MDCS_GATEWAY_SERVICE" returned 1 (generic error)
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Stopping service service:MDCS_GATEWAY_SERVICE
Mar 28 07:25:15 BA-GW1 clurgmgrd: [4751]: script:MDCS_DBLOADER_SERVICE: status of /etc/init.d/dbloader failed (returned
1)
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: status on script "MDCS_DBLOADER_SERVICE" returned 1 (generic error)
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Service service:MDCS_QUERY_SERVICE is recovering
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Stopping service service:MDCS_DBLOADER_SERVICE
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Service service:MDCS_GATEWAY_SERVICE is recovering
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Recovering failed service service:MDCS_QUERY_SERVICE
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Service service:MDCS_DBLOADER_SERVICE is recovering
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Recovering failed service service:MDCS_GATEWAY_SERVICE
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Service service:MDCS_QUERY_SERVICE started
Mar 28 07:25:15 BA-GW1 clurgmgrd[4751]: Recovering failed service service:MDCS_DBLOADER_SERVICE
Mar 28 07:25:16 BA-GW1 clurgmgrd[4751]: Service service:MDCS_GATEWAY_SERVICE started
Mar 28 07:25:16 BA-GW1 clurgmgrd[4751]: Service service:MDCS_DBLOADER_SERVICE started

Attach also is a sample script of a script is use to start the application.

Re: Red Hat Cluster suite 5 question

Ivan Ferreira — Mon, 31 Mar 2008 20:14:03 GMT

script:MDCS_QUERY_SERVICE: status of /etc/init.d/queryService failed (returne
d 1)

You must identify why the status part of the script is not returning 0. Run the script manually outside red hat cluster suite and ensure that you always get exit code 0.

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Tue, 01 Apr 2008 05:06:43 GMT

Sorry posted the old script. Here attach is the new script that is use with cluster.

Re: Red Hat Cluster suite 5 question

Alex Ng Chong Yau — Tue, 01 Apr 2008 06:46:01 GMT

My script actually locate in /etc/init.d. When it run with cluster the service restarted by itself after 30mins or less. Have try to run the script without using the cluster, it seems everything is normal and it does not stop or restart and there is no error. Took out the script from /etc/init.d, place it in another directory and run with cluster the service restarted by itself after 22mins.