topic Re: Cluster Heartbeat connection by a cross cable in Operating System - HP-UX

Cluster Heartbeat connection by a cross cable

Kaushlesh Kumar — Wed, 10 Jun 2009 10:56:12 GMT

Dear Sir,

I need a clarification that can we connect RAC through cross cable.

I have two DB server on which cluster implemented . there are four NIC card 2 are assigned prvate IP for Data and 2 are assigned VIP( 10.0.0.1) for cluster.

So can we make connectivity of this VIP of one server to another server

Re: Cluster Heartbeat connection by a cross cable

Michal Kapalka (mikap) — Wed, 10 Jun 2009 11:01:20 GMT

hi,

cross cable is the same, if you are using the normal switches between. We have MC/Servicegueard with 2 HeartBeat connection with the cross cable.

mikap

Re: Cluster Heartbeat connection by a cross cable

Kaushlesh Kumar — Thu, 11 Jun 2009 04:35:09 GMT

Sir,
Presentlay i have a 100 mbps switch to connect these two DB server over heart beat .

But i want to connect two DB directly from 1st server's NIC to another server's NIC directly with cross cable . i want to remove 100 mbps switch.

Can i do ....

Re: Cluster Heartbeat connection by a cross cable

Michal Kapalka (mikap) — Thu, 11 Jun 2009 05:16:08 GMT

hi,

you can do it, the cross cable can be used for HB connection betveen 2 cluster nodes.

mikap

Re: Cluster Heartbeat connection by a cross cable

Sajjad Sahir — Thu, 11 Jun 2009 05:34:39 GMT

Dear Friend

we are using two cross cables for reduncy and for heartbeat

thanks and regards

Sajjad Sahir

Re: Cluster Heartbeat connection by a cross cable

Suraj K Sankari — Thu, 11 Jun 2009 05:51:50 GMT

HI,
>>But i want to connect two DB directly from 1st server's NIC to another server's NIC directly with cross cable .

Yes, you can use cross cable between these two DB server.

Suraj

Re: Cluster Heartbeat connection by a cross cable

sujit kumar singh — Thu, 11 Jun 2009 11:36:32 GMT

hi advantages of using the croos-over-cables for heartbeat:
1) one component that is switch is removed from the list of potential failing devices.

disadvantages:

1) you cant extend the number of nodes bveyond 2.
2) with a switch in place at least using the LED indications you can find which Servers LAN is faulty.

regards
sujit

Re: Cluster Heartbeat connection by a cross cable

Mel Burslan — Thu, 11 Jun 2009 11:44:17 GMT

another disadvantage:

you can not monitor the health of this heartbeat connection, whereas most oribably all your switches are being monitored for uptime.

By replacing a couple of network ports, you are shifting your trust from a hundred thousand dollar worth of equipment (I am assuming you are using enterprise grade Cisco switches by this price) to a $2 worth cable. Think what will happen when people start pushing these cables to the edges of the cabinet to make the cabinet look good. They will start breaking and you will not be able to notice, until one day, when your cluster will take a dump for no other reason than losing a heartbeat.

I strongly discourage you from doing this. If you are willing to shell out big bucks to have serviceguard license to maintain your business continuity, I am sure you can afford to spend a couple of hundred bucks on a couple of switch ports.

My 2 cents worth advice

Re: Cluster Heartbeat connection by a cross cable

Steven E. Protter — Thu, 11 Jun 2009 14:12:33 GMT

Shalom,

I've done this in my home lab.

It works. You can monitor the connection with a ping script.

When possible, I run a heartbeat through a private network. Usually I try and cajole network group to give me an old cisco switch for the purpose.

SEP

Re: Cluster Heartbeat connection by a cross cable

George Denyer_1 — Thu, 11 Jun 2009 16:36:31 GMT

Although its possible to use a crossover as a heartbeat link it reduces all heartbeat failure conditions to the same result: both nodes lose link on the heartbeat NIC. With a switch in the middle if node A's NIC fails node B wont communicate with node A but it will still have link on its NIC so it will know the problem is on node A.

Re: Cluster Heartbeat connection by a cross cable

TTr — Thu, 11 Jun 2009 16:56:36 GMT

> But i want to connect two DB directly from 1st server's NIC to another server's NIC directly with cross cable . i want to remove 100 mbps switch

You must use redundant heartbeat connections and set them up in the cluster configuration. You should use at least two connections from the following options
1. dedicated HB direct connection (crossover)
2. dedicated HB on HB LAN switch
3. HB over the DATA1 IP
4. HP over the DATA2 IP

With only one heartbeat connection, a simple cable failure, direct crossover or through a switch, will cause a failover.

Re: Cluster Heartbeat connection by a cross cable

Duncan Edmonstone — Fri, 12 Jun 2009 04:59:54 GMT

Hi,

whilst crossover cables are supported for Serviceguard for heartbeat with the limitations others have indicated, they are most definatelt NOT supported by Oracle for RAC cluster interconnects.

Here's an excerpt from Oracle's RAC FAQ:

------------------------------
Is crossover cable supported as an interconnect with RAC on any platform ?

NO. CROSS OVER CABLES ARE NOT SUPPORTED.
The requirement is to use a switch:

Detailed Reasons:
1) cross-cabling limits the expansion of RAC to two nodes
2) cross-cabling is unstable:
a) Some NIC cards do not work properly with it. They are not able to negotiate the DTE/DCE clocking, and will thus not function. These NICS were made cheaper by assuming that the switch was going to have the clock. Unfortunately there is no way to know which NICs do not have that clock.
b) Media sense behaviour on various OS's (most notably Windows) will bring a NIC down when a cable is disconnected.
Either of these issues can lead to cluster instability and lead to ORA-29740 errors (node evictions).

Due to the benefits and stability provided by a switch, and their afforability ($200 for a simple 16 port GigE switch), and the expense and time related to dealing with issues when one does not exist, this is the only supported configuration.

From a purely technology point of view Oracle does not care if the customer uses cross over cable or router or switches to deliver a message. However, we know from experience that a lot of adapters misbehave when used in a crossover configuration and cause a lot of problems for RAC. Hence we have stated on certify that we do not support crossover cables to avoid false bugs and finger pointing amongst the various parties: Oracle, Hardware vendors, Os vendors etc...
------------------------------

So it's not that it doesn't work, but rather that Oracle don't support it. So you can use at your own risk, but don't expect too much help from Oracle support when diagnosing network related cluster issues...

HTH

Duncan