One cluster, two subnets - help!

[NSMD BT UNIX]

[Sorry for the previous empty message - hit return too early]

Hi SG gurus,

We want to configure a cluster with 10 nodes, of which half are in one subnet (subnet 1) and the other half in another subnet (subnet 2). The two subnets are for client data only, a separate network is used for SG heartbeat traffic.

I was able to successfully run cmquerycl to scan all nodes and generate a config file. Now I run cmcheckconf to verify the configuration and this is what I am getting (the command is launched from a node in subnet 1):

Error: Unable to establish communication to node node_s2: 19

where node_s2 is a node is subnet 2. If I launch the command from a node in subnet 2, I get the same error on a node in subnet 1. If I comment out in the config file all nodes in subnet 2 and launch cmcheckconf from subnet 1, it works fine (and inversely, when the command is launched from subnet 2 with the nodes in subnet 1 commented out).

My question is thus: is it possible to have an SG cluster that spans multiple subnets for client data?

The SG version is A.11.15 and the nodes are Superdome vPars. I have a case open with HP support on this. Any help is appreciated.

Thanks in advance,
Dmitri

[Michael Steele_2]

General rule of thumb for building ServiceGuard: Stay within one subnet.

Heartbeats go out through every lan and are tested during 'cmquerycl' with 'linkloop MAC'.

MAC's are going to travel through a router well.

[Michael Steele_2]

MAC's ARE NOT going to travel through a router well.

Sorry, still working on my first cup.

[Geoff Wild]

You will have to have lan cards configured on each server into each subnet to make this work....so, you will need at a minimum for each server:

2 Lan cards for Subnet 1
2 Lan cards for Subnet 2
1 Lan Card for Heartbeat (Subnet 3)

I mention the heartbeat as you will need a network that all servers see at the same time...

Rgds...Geoff

[Sanjay_6]

I would suggest you get atleast three cards on each system. 1 for subnet 1, other for subnet 2 and let the third be failover. Let both the cards (subnet 1 & 2) carry the heartbeat, so you have multiple heartbeats.

You can configure the floating ips for the packages in different subnets.

Hope this helps.

Regds

[John Poff]

Hi,

We have a cluster built with this configuration. Geoff has it exactly right. You'll need multiple LAN cards that can see each subnet. It will work. As always with MC/SG, get the LAN configuration correct and SG will be happy.

JP

[NSMD BT UNIX]

Thanks everyone for your replies.

One thing I forgot to mention: the packages are configured in such a way that for each primary node there is one failover node *in the same subnet* (never in another subnet).

Referring to what Geoff said, my goal is to have:
- for nodes in subnet 1: 2 lan cards in subnet 1 (no cards in subnet 2)
- for nodes in subnet 2: 2 lan cards in subnet 2 (no cards in subnet 1)
- for all nodes: 1 lan card on subnet 3 for the heartbeat.

Can this configuration work given that there is never a failover between subnet 1 and subnet 2 (and inversely)?

[Rita C Workman]

Or you try using the Continental Cluster software for MC/SG. Since the failover to a different subnet is what it does.

Just a thought,
Rita

[James Randall]

You might want to rethink your plan and create two seperate clusters.

This would eliminate the cross-subnet issue you are talking about.

If you will not be trying to cross a package from subnet1 to subnet2 (or vice versa).....Why does it need to be one cluster?

[Marcel Kuiper]

The reason we are implementing one cluster instead of two or more is licensing costs (we are using MetroCluster with EMC/SRDF and the license is on a per-cluster basis).

At the time of the original posts, I had multiple problems in my configuration:

1) Not all network interfaces were declared in the cmclnodelist file. The service (client data) interfaces were declared but the heartbeat interfaces were missing.
2) Linkloop problems between some service interfaces and their standby interfaces.

As soon as I resolved these issues, everything worked fine. I was able to compile and run the cluster and all the packages.

So, to resume, it *is* possible: two subnets inside of one cluster, provided that packages are not crossed between the two subnets.

Thanks everyone for your replies :-)

[Brian Markus]

We use APA (Auto Port Aggregation) I have lan100, etc.. two cards meshed into one for redundancy on subnet 1, and the same on subnet 2. Total of 5 network connections per machine. 2 = subnet1 2=subnet2 1=heartbeat. It's been working great for 2 years now. It's kind of a pain to setup at first, but once it's done it's rock solid.

Hope this helps.

-Brian.

[NSMD BT UNIX]

Brian, does this mean that you do not have any standby interfaces configured in the cluster?

Seems to me like a good alternative.

[Brian Markus]

On that machine I felt confident in APA to cover redundancy. We have had a card fail and have absolutely no down time.

Here is part of my cmclconf.ascii file.

# Link Aggregate lan100 contains the following port(s): lan3,lan1
# Possible standby Network Interfaces for lan100: lan0.

I personally have been saved several times by APA thus far, and build all my boxes with it when I can.

Good luck

-Brian.

[Pramod_4]

Dmitri,

I was wondering whether you can post your cluster.ascii file here.

There is no reason why you can not have a cluster with two subnets and a dedicated heartbeat lan.

Thanks,
Pramod