Adding a node in VMS cluster

[Malkeet_1]

Hi

We have 2 node GS1280(M16)OpenVMS 7.3-1 cluster using cross over cable as cluster interconnect. This option was used as both the nodes were adjacent in the Data centre. A third GS1280(M32) needs to be added in the cluster.

What would be the best options for a cluster interconnect.

1. non routable VPN, seperate switch or any other option.

The current cluster is hosting a 24x7 clinical application.

Thanks in advance

Malkeet

[Heinz W Genhart]

Hi Malkeet

welcome to the OpenVMS Forum.

We have a similar configuration running. We are using GB interfaces on 4 nodes, connected to a separate switch for cluster communication.

That works fine for us.

Regards

Geni

[Robert Gezelter]

Malkeet,

A separate switch (or a hub) for private cluster communications would be ideal.

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

Classic computer answer: "It depends."

Switched GbE, minimally.

What is the current network traffic on your cross-over LAN? MONITOR or (better) T4 can get you this data. Both are free.

This configuration could be anything from a consumer-grade GbE switch on up to some rather more serious equipment, depending on your current traffic and your current and upcoming expansion.

There's also the storage I/O traffic, and how much of that is going to be local and how much of that will be remote -- what part of the I/O traffic ends up on the GbE, and how much locking is going on.

Stephen Hoffman
HoffmanLabs

[Andy Bustamante]

Your cluster interconnect should be on a network switch or hub that share no single point of failure with the service LAN.

Specifically, make sure you don't allow your networking team to create a VLAN for cluster traffic on the same switching your network traffic is using. It's a common recommendation, but you're creating a single point of failure. If both LANs are unavaile due to an equipment outage, your cluster will hang.


Andy

[Jon Pinkley]

Malkeet,

It's hard to find something that is as fast or reliable as a crossover cable :-)

However, since that isn't an option with more than two nodes, you probably want a dedicated switch.

Do you know what the types of traffic are using SCS now? If it is primarily DLM (distributed lock manager) traffic, the primary concern will be latency. If it is data (MSCP serving), then you will want to make sure that the switch supports jumbo frames >= 9000 bytes. Likewise you will want to make sure your SYSGEN parameter NISCS_MAX_PKTSZ is high enough to take advantage of the jumbo frames. If you don't override in your modparams.dat, AUTOGEN will normally do the right thing.

While somewhat dated, this c.o.v thread is still worth a read.

http://groups.google.com/group/comp.os.vms/browse_thread/thread/c253c0310a34aca0/420d011995c8cee7

Have fun,

Jon

[John Gillings]

>It's hard to find something that is as
>fast or reliable as a crossover cable :-)
>
>However, since that isn't an option with
>more than two nodes, you probably want a
>dedicated switch.

If all systems are in the same room, and you have sufficient network adapters you CAN connect three nodes with cross over cables.

The minimal requirement for a cluster is that each node should have a direct path to each other node. Thus, if you have 2 adapters on each system which are available for cluster traffic, you can connect them in a triangle topology, each node connected to the other two via a cross over cable.

You could continue adding nodes until you run out of adapters/slots.

On the other hand a switch is much more scalable, and if you have two adapters available, you might be better off using them redundantly - connect each adapter to a private switch so there are two independent paths from each node to each other node.

The important thing to remember about cluster interconnections is that you don't really need to do much more than connect the systems. For most applications, cheap commodity parts will more than satisfy performance requirements. The more paths, the more failures you can tolerate and (at least >V8.2) the more traffic.

[Malkeet_1]

Thanks you all for your input.

I think it will probabaly come down to a non routabel VPN on a network switch at this site. Will push for redundant switchs for cluster inter connect.

For reconfiguration plan would be to move the existing nodes from cross over cable to the switch by shutting the cluster down and plugging 2xGb interfaces from each node.

Reboot the cluster.

And a later stage add the third node via cluter_config.com.

Or this any thing to be aware of.

Thanks in advance

Malkeet

[Malkeet_1]

Should have also mentioned in the earlier post that we a redundant switch being used for Tru64 Cluster. Will it a good idead using these for VMS cluster inter connect.

Malkeet

[Robert Gezelter]

Malkeet,

Using a separate switch as the primary path for cluster communications is almost always a good idea.

At this instant, I do not offhand remember the details of when the failover support for multiple cluster communication paths appeared. If you have an additional Ethernet adapter, you may be able to accomplish the reconfiguration without taking the cluster down. Caution obviously applies, do not do this during a busy time.

- Bob Gezelter, http://www.rlgsc.com

[Jan van den Ende]

Malkeet,

like Bob said, this can be accomplished even without cluster shutdown.
Each node WILL have to shut down if you need to add adapters, but that can as easily be done while the other node is running.

Ubtil nowm I did not see any suggestion to upgrade your OS. Please, DO go to a supported config!
If there are compelling reasons to stay at V7, then go V7.3-2 ( + recent UPDATE ECO ), but if there is no serious reason NOT to, then go to 8.3.
Various interesting new features, but above that, much better performance. And since you are adding anotherGS1200, THAT _must_ be interesting for you!

hth

Proost.

Have one on me.

jpe

[Andreas Vollmer]

Hello Malkeet,

I agree with the previous advice of the community but...
Please use a network switch and NOT a hub.
We made a bad experience with the hub (it was an old model and it it was not able to deal with 100 Mb FD and the auto negotiation ended with 10 Mb HD).
In order to spare you that painful handling please use a switch instead.
Due to the importance of the cluster it would make sense to save some small $ in exchange for big trouble.
Be also aware to have some important cluster parameter in sync on all three nodes - such as the LOCKWRT parameter (dealing with the lock tree) etc.
Kind Regards
Andreas

[Andreas Vollmer]

Hi again,

The mentioned parameter is called LOCKDIRWT and not LOCKWRT -- Sorry.
- CLUSTER_CREDITS should also be checked.

The reason why I mentioned the parameters is we experienced performance problems in one cluster. It turned out that some important cluster parameter where not equal or in sync and that caused the problems.
BR
Andreas