Remove VAXes from Cluster and FDDI

[Art Wiens]

Right off the bat I'll say I have 0 experience with FDDI and I have little knowledge of the physical environment in question.

My customer wants to remove the two VAXes from their 4 node cluster leaving two Alpha's. Votes, expected votes and a quorum disk will be adjusted/configured. The "quirk" (to me) is all 4 systems have an FDDI connection through some sort of FDDI hub/switch I suppose. Is there anything to be done on said hub/switch when the VAX FDDI cables are removed? Any kind of terminator or such on open interfaces required? Maybe just rubber plugs so some unsuspecting operator isn't "blinded by the light"?

The other curious display I see is in SHOW CLUSTER (see attached). It says the Alpha's have a PMA0 circuit. Memory Channel? Or did FDDI just use the same driver? VMS v7.1-1H1 ;-(

Cheers,
Art

[Hoff]

You might be interested in "OpenVMS Tips: Adding or Removing Nodes from a Cluster", over at http://64.223.189.234/node/169

Yes, it appears your Alpha systems do have MC. You'll probably see it enabled via the MC_SERVICES_P2 parameter.

If the Alpha systems can be retrofit with GbE, I'd be tempted to remove FDDI entirely. And given they likely have 100Mb and they do apparently have MC, I'd probably remove FDDI.

FDDI is usually wired as a pair of rings, and daisy-chained around all the controllers. As for the FDDI cabling, you'll be capping off the disconnected FDDI controllers (with boots), and re-organizing the FDDI rings to include just those systems you want to remain in the ring. If you are using FDDI at all. Or you can conceivably split the rings into two.

Also remember the SET NOINTERFACE and SET CONFIG NOINTERFACE and the equivalent NCP or NCL commands for whichever DECnet is in use, if any.

Stephen Hoffman
HoffmanLabs

[Art Wiens]

I can't believe I could question Hoff (!!), but are you sure about the presence of memory channel? I have attached another file showing the output from analyze/system show lan. Would I not see something there? Or alternatively, should I not see three circuits in the show cluster (lan, fddi, mc)? Also in Decnet V, I see only FDDI-0 and CSMACD-0.

Any other way of determining if MC really is there? It's 3000 km away so I can't just "have a look".

I have 0 exp w/FDDI, I have less than that on MC! ;-)

Thanks,
Art

[Bill Hall]

Art,

Memory channel isn't a LAN device. $SHOW DEVICE MC. Does CLUE CONFIG exist under SDA at that old version of VMS?

Bill

[Art Wiens]

Ok, now I'm quite confused. Memory channel does indeed seem to be there (you'ld think this might have come up in conversation somewhere in the last 4 years!) but it's /NOAVAILABLE, but SHOW CLUSTER says it's in use? I thought all nodes needed a common medium for cluster communications? There wasn't MC for VAX 4000's was there?

Yes CLUE CONFIG exists in 71-1h1 ... see attached.

Art

[Hoff]

I interpreted "circuit" as a cluster virtual circuit; as the existence of a cluster-related device PMA0.

$ SHOW DEVICE PM
$ SHOW DEVICE MC

Or check the parameter.

Or SDA> CLUE CONFIG

If you meant it as a circuit as in the DECnet context, then SHOW KNOWN CIRCUIT CHAR or such. (Though you can't boot over MC, most everything else can traverse MC. IIRC.)

[Hoff]

re: mc devices as /noavailable

http://h71000.www7.hp.com/wizard/wiz_4296.html

Depending on the vintage, SHOW DEVICE doesn't know the status of the MC device, and SDA does.

Yes, you have MC.

Your FDDI device is FW.

Most (but not all) Ethernet devices are E*, while FDDI was to be F*. There are some odd-ball devices -- stuff E* that wasn't Ethernet, or stuff that was not-E* that was Ethernet -- but that was the general prefix assignment theory.

If you are interested, you can contact me offline, and we can discuss the hardware and software in rather more detail, and one of us can post back a summary here. These little tiny text boxes really slow things down. IRC or email or such can be way faster...

[Art Wiens]

Sorry, I must just be a little denser than normal today ...

Things I know: the Alpha's have ethernet, fddi and memory channel and the VAX's only have ethernet and fddi.

Things I think I know: in a VMS cluster, all members have to have a common communication path for SCS traffic.

Things I don't understand: how can MC be the "active" circuit (NUM_CON) displayed by SHOW CLUSTER? Why don't I see a reference to fddi in that display? One of the VAX's has been booted back into the cluster and the display has not changed.

Art

[Hoff · Accepted solution]

>>> Things I know: the Alpha's have ethernet, fddi and memory channel and the VAX's only have ethernet and fddi.<<<

Your proximity to the configuration allows you a better view into the connectivity.

>>> Things I think I know: in a VMS cluster, all members have to have a common communication path for SCS traffic.<<<

False.

Each node has to have a direct path to each other node, per the rule of total connectivity.

There is no requirement that there be just one path between nodes, nor is there any requirement to have one path common to all nodes.

Ethernet or FDDI makes meeting total connectivity easier, but four nodes can be wired with four point-to-point DSSI links or with two tri-node DSSI links, for instance.

The reason you likely have FDDI is because the VAX nodes did not -- at the time this configuration was implemented -- have fast Ethernet controllers available. Nemonix Engineering has provided 100 Mb NICs for certain of the older hardware, as stock DIGITAL networking gear for VAX systems was somewhat lacking in certain features.

[Hoff]

Things I don't understand: how can MC be the "active" circuit (NUM_CON) displayed by SHOW CLUSTER? Why don't I see a reference to fddi in that display? One of the VAX's has been booted back into the cluster and the display has not changed.

The VAX nodes were participating via LAN, while the Alpha nodes were participating via LAN and MC. Whether or not FDDI was in use is not (yet) in evidence, nor whether this FDDI used a classic ring -- or a switch or concentrator. At this number of nodes, I'd tend to assume "ring".

As for circuits, SHOW CLUSTER/CONTINUOUS with the ADD CIRCUITS or other similar displays, or the various SCS commands available in more recent SDA implementations. Older SDA was adequate in its display capabilities, but Mr. SDA has gone quite gonzo adding stuff into SDA in newer releases.

[Martin Hughes]

As I understand it, SCS traffic will simply go via the fastest path available. The path will be chosen in the order listed below.

1. Memory Channel (MCDRIVER)
2. CI (PADRIVER)
3. DSSI (PADRIVER)
4. FDDI/Ethernet (PEDRIVER)

In the case of 4. PEDRIVER will compare latency and choose the fastest path.

[Art Wiens]

Well thanks everyone, it was a good learning day for me! I read the Cluster guidelines manual and realize that I've missed the "sublety" of the difference between the types of interconnects -

Cluster interconnects (node to node only)
Ethernet (10/100, Gigabit)
Asynchronous transfer mode (ATM)
FDDI (Fiber Distributed Data Interface)
MEMORY CHANNEL
Shared Memory CI (SMCI) (Galaxy instance to Galaxy instance)

Shared-storage interconnects (node to storage only)
Fibre Channel
SCSI (Small Computer Systems Interface)

Both node-to-node and node-to-storage interconnects
CI (computer interconnect)
DSSI (Digital Storage Systems Interconnect)

And that both ethernet and fddi use the PEDRIVER ... SDA is showing me a virtual circuit.

So now with my new knowledge of the hw config having MC, and clearer picture of how SCS can work, perhaps it's ok to yank fddi altogether. The nic and fddi are both 100Mb and it theory says it will use MC anyways. Ethernet can remain as a redundant SCS path but there is no need to MSCP serve the HSx disks to "anyone" at 100Mb any more.

Thanks again,
Art

[Anton van Ruitenbeek]

Art,

To add a small issue to it:
FDDI is already done by a concentrator, so you have only to remove the two VAXes. As Steve already mentioned, FDDI is a dual system. You only will see messages in OPCOM that you remove next/previous node and automaticly will see another next/previous node.
FDDI and NIC are theoreticly the same speed. Because of the nature of FDDI, this will be effective about 80Mb and NIC's isn't realy to tell. This is depending on the brance of routers you have and ofcourse the traffic. I know a big problem Cisco has, is that when you have a flame of traffic, Cisco drops _ALL_ other traffic than IP. This is crusial for clusters because they need SCS. You also using MC and this is the prefered path because its the fastest. Only also the shortest expand.
Whe didn't use MC because its to expensive for the need and the distance is to short. We have a cluster over more then 7 km. Thats wy we have FDDI (and Digital concentrators and fallback to 100Mb NICs (with unfortunaly Cisco concentrators).

AvR