Restrict cluster communications to subset of NIC

[Richard W Hunt]

Running OpenVMS 7.3-2, patched through 1-Feb-2008, running TCPIP services v 5.4 ECO 7. We have a four-channel 100 Mb ethernet card per box. No DSSI, SCSI, or MC interconnects.

We are connected to at least two Ethernet-type networks that are experiencing frequent frame check/alignment issues. Our database guys confirm that we are using the channels that are noisy. We ALSO have two other channels that show up as pretty good (stats-wise), but the cluster apparently isn't using them.

How can I tell cluster communications services to avoid two of the four channels as potential problems? The channels that lead to the rest of the world should not be used for local traffic if two perfectly good local connections are available.

We also have two KGPSAs per box that are used for HSG-class traffic. Can I define a way for these to participate in the cluster traffic?

Before anyone asks: I cannot completely turn off the bad channels, cannot fix them myself, and don't control them. The two good channels are much easier to control. I have reported the problem to our networks team but this is a US government site. It takes way too much paperwork to get ANYTHING done fast when another agency is involved. The two bad subnets, in this case, are not ours.

I've been reading manuals all morning but haven't gotten very far. It seems that what I want to do is defined in about six or eight manuals in a distributed fashion. Which doesn't mean it isn't there, I just haven't found it yet.

Also, part of the problem is that after a "corporate shakeup" I inherited the cluster already configured and am having just a bloody WONDERFUL time trying to figure out what was done to it. So if my questions seem a bit confused, it is because I wasn't the one who put it together and the person who DID is no longer here. Nor can I find her notes.

I'll take any help I can get including someone pointing me to some relevant part of the manuals.

[Robert Gezelter · Accepted solution]

Richard,

Have you set the priorities using SCACP?

- Bob Gezelter, http://www.rlgsc.com

[Richard W Hunt]

Bob, no I haven't. But I'll look that up now.

[Hoff]

Set your preferred path (SET PREFERRED_PATH and related, and particularly SCACP) to cause the cluster traffic to be "costed" over to the stable network paths, and (if you want, as it appears to be working) leave the unstable paths enabled but more expensive (as a fallback).

You can enable packet verification in SCS, but it's comparatively compute-expensive. (Yes, you guessed it. There's a cluster system parameter. NISCS_PORT_SERV. Or you can use the SCACP command SET VC/CHECKSUM. This is at the same level as the SCS data compression knob.)

And no, there's no way to run host-to-host cluster communications traffic over a Fibre Channel SAN connection.

It feasible to build a stack which can provide both. With some (comparatively expensive) switches, you can potentially bridge it all over an IP network; you can run FC and SCSI over an IP connection.

Stephen Hoffman
HoffmanLabs LLC

[Bill Hall]

Richard,

We use two dedicated GigE NICs in each cluster member that are dedicated to SCS traffic. The NICs are wired to two standalone ethernet switches similar to using two redundant Memory channel hubs in a cluster.

In SYCONFIG.COM I stop all SCS traffic on our NICs used for DECnet, LAT and tcpip traffic.

$mcr scacp stop lan_device LLA
$mcr scacp stop lan_device LLB

Bill

[Robert Atkinson]

Or the old way, if you're so inclined :-

$ ! Added by RAAto ensure SCS traffic does not go across this disconnected port
$ !
$ MC LAVC$STOP_BUS EIA
$ MC LAVC$STOP_BUS EIB
$ MC LAVC$STOP_BUS EWA
$ !

[Richard W Hunt]

Thanks, guys. I didn't set up this cluster so didn't know what tools were used. This makes my life a bit easier now.

Or, as one politician in my home state was overheard to say about inheriting something from a previous administration, "It's not my baby but I've got to rock it."