SG Heartbeat in Extended Distance Cluster

[Greg Martin]

We're in the process of moving an existing cluster from a local to extended distance (campus) configuration, and have some questions about the recommended network architectures.
The most recent pertinent document seems to be "Designing Disaster Tolerant High Availability Clusters", Part no B7660-90013, March 2003. We are looking at two sites 3-4 miles apart, with SAN based storage over switched fabric, two arrays replicated using mirrordisk. We're currently in the process of upgrade to a "highly available" gigabit network using all switching internally, linking switches on each campus by fibre links, subnet separation by vlan.

The recommeded architectures require two separate etherlans for heartbeat with separate physical pathing, but do not deal with the question of network data. (Heartbeat designs for ethernet or fddi seem to require 2 separate links between campuses; dwdm designs seem to prefer 2 dwdm connections, but accept 1 dwdm to dwdm link as long is it has ability to failover to separate physical routes).
One possibility for us would be to request two separate vlans over this switched network. Another reasonalble alternative would be for us to add a dedicated heartbeat connection with 100bt-fibre-100bt in addition to the planned switched, and use the public network for heartbeat/data. Another proposal has us with two dedicated gigabit heartbeat connections, using the public network for data.
Traditional SG architecture would seem to imply that we use one link for heartbeat/data, and another as dedicated heartbeat. This might be based on separate vlans, or on public network/backend heartbeat network.
The white paper doesn't address network data. I could infer it's acceptable to run data/heartbeat over both links because of high availability and high bandwidth (and this is attractive as there is some reason for us to consider multihoming the hosts, particularly in the phasing in of the new network...), or, that 2 dedicated heartbeat networks are recommended. Or that the question is not addressed, and the old 1 mixed, 1 dedicated recommendation still stands.

Any thoughts or other resources you can recommend would be appreciated.

[melvyn burnard]

You can certainly run data as well as heartbeat across both lans, and there is no reason why you should not. It all depends on how you lay out the network and the reliability of hte paths you will use. I would certainly recommend you have each lan backed up by a standby which goes via the alternate route, thus you will not lose a complete link if someone digs up htat route.

[Stephen Doud]

Hi Greg,
Architecting a potentially expensive extended-distance cluster via a Forum has it's limitations. If you would like to have an HP rep help out in this area, please email me contact details.

Stephen.Doud@hp.com

[Greg Martin]

melvyn - it's not a question of what will work, but of what may be the minimum "supported" configuration, what the intent is and why.

stephen - I sent you contact information by email... Haven't heard anything back.

We'll be talking in more detail with the app vendor, HBOC, next week, and get tehir thoughts on the network config. Thanks.

[Justin Willoughby]

Greg,

What have you found out? We are looking to do the same thing. We also use a McKesson product on one system we want to use with MC/SG between two servers in two buildings. Any info would be helpful.

- Justin

[Greg Martin]

As above, it's not a question of what's required, but what's advisable and affordable.

On the existing 2 node cluster, with serial heartbeat and data net heartbeat, but no dedicated heartbeat network, we had a failover recently due to data congestion. Both primary interfaces are plugged directly into the backbone router which apparently was overloaded. A rogue server/app had just been connected which quadrupled our network load!

Anyway, since we have the existing fibre runs and the $, we've decided to go with a separate, redundant heartbeat network, with a gigabit switch to fibre to gigabit switch topology, along with a quorum server. On the front end, redundant switches for primary and failover lan cards runs to the public network.

We'll be adding additional clustered servers, so the separate heartbeat network cost will be spread over a number of applications. Actually, tho I'm skeptical about the need for this, we'll be clustering an app on two Sun servers using Veritas clustering, and I'm told they "require" two dedicated heartbeat networks, so we may get yet another dedicated heartbeat network.

Jason from HBOC will be out next week to do a laying on of hands so we have McKesson's blessing (makes management happy...).