DECNET IV and UCX 4.2 - ECO 5 issue

[Russ Carraro]

Customer is replacing several VAX clusters with individual Charon VAX emulation servers.  The Charon Servers are behind new firewall vault that allows DECNET traffic within the vault but not into or out of the vault.  The original clusters are outside the vault.

Charon VAX is running OpenVMS 6.2 or 7.1, DECNET IV and UCX 4.2 - ECO 5.  These Charon VAXs are the only VAX or ALPHAS in the vault.  When I boot the Charon VAX starting DECNET and then UCX, they both seem to start.  I see circuit and line in NCP and LANCP show config shows the device.  However, when I start both DECNET and UCX I can not telnet or ping the Charon VAX from outside the vault.  If I boot starting just UCX I can ping/telnet to the Charon VAX from outside the vault.

I need help figuring out what the problem is and how to correct it.  Any ideas are apprecited.

[Volker Halle]

Russ,

when starting DECnet-IV, the MAC address of the network interface gets changed to the DECnet MAC address: AA-00-04-00-xx-yy (where xx-yy are derived from the DECnet Phase IV address).

When starting DECnet first, can you SET HOST and TELNET/PING between the CHARON-VAX systems inside the 'vault' ? If so, CHARON-VAX is o.k.

The firewall may only be allowing the LAN hardware interface MAC addresses to pass, but may block the packets after the MAC address has changed to the DECnet MAC address.

Use ANALYZE/SYS SDA> SHOW LAN/FULL to obtain the DECnet MAC adresses.

Volker.

[Richard Brodie_1]

When you start DECnet it will change the physical MAC address of the (emulated) card.

You need to be careful that you aren't duplicating the address on a bridged network, and that you haven't got any address based firewalling.

Normally you won't be able to start DECnet if the address is already in use on the LAN but your firewall probably filters out loopback test packets...

[Russ Carraro]

Volker

I booted two of the Charon VAXs (there are three) starting DECNET and UCX.  On both systems I could "set host" to itself (either 0 or nodename) and ping the node.  Trying to "set host" or ping the other node failed.  I did the analyze/sys: is this what I should see:

    Press RETURN for more.
       -- EXA2 60-03 (DECNET) Unit Information 18-APR-2012 16:36:20 --

LSB address                 BBBF16C0    VCIB address                00000000
Packet format               Ethernet    Error count                        0
Device buffer size              1500    LAN medium                    CSMACD
Maximum buffer size             1498    Eth protocol type              60-03
Hardware buffer quota             16    802E protocol ID      00-00-00-00-00
Receive buffer quota           15040    802.2 SAP                         00
Allow prom client                 ON    802.2 Group SAPs         00,00,00,00
Promiscuous mode                 OFF    Maximum header size               16
802.2 service                    OFF    Hardware address   00-50-56-96-00-0E
Data chaining                    OFF    Physical address   AA-00-04-00-D0-6A
Padding mode                      ON    Can change address               OFF
Automatic restart                OFF    Access mode                EXCLUSIVE
CRC generation mode               ON    Controller mode               NORMAL
Maintenance state                 ON    Rcv buffs to queue                10
P2 parameters               00374395    Starter's PID               00010012
All multicast mode               OFF    Creator's PID               00010012
Rcv buffer quota               15040    LSB size                        3496

Since thses are the first nodes behind the firewall using DECNET to I need to create some sort of DECNET networ or make one of these a routing node?

[Volker Halle]

Russ,

in a DECnet-IV network on a local LAN segment, you don't need DECnet routers. End nodes can communicate directly by using the DECnet MAC address to establish connections to the other DECnet nodes. Commands like SET HOST 0 or UCX PING LOCALHOST do not even pass the LAN interface, so these commands only test, if DECnet or UCX on the local node are actually up and running.

Did you check, that DECnet node addresses are correctly configured ? Did you copy the system disks of the original systems or did you install OpenVMS VAX from scratch on those 3 CHARON-VAX systems ?

The easiest way to check, whether you really have LAN connectivity in your local LAN segment, is to use the LAT protocol:

$ @SYS$STARUP:LAT$STARTUP

$ MC LATCP SET NODE/CONNECTION=BOTH

then wait a couple of minutes and issue

$ MC LATCP SHOW SERVICE

You need to enable LAT on all your CHARON-VAX OpenVMS nodes for this test.

LAT uses multicast messages to announce it's services. So as long as the local LAN correctly works, you'll see the LAT services (service name = node name) offered by the other nodes after a couple of minutes.

If you don't see the LAT service announcement messages from the other CHARON-VAX nodes in your LAN segment 'inside the vault', then you need to troubleshoot your CHARON-VAX configurations. If required, I can also help you with that task...

Volker.

[Russ Carraro]

Just found out the customer did move another Alpha cluster with three nodes into the vault a while back and DECNET is working for that cluster, i.e. set host from one Alpha to another is working.  I get a timeout if I try to set host to the Alpha from the Charon VAX.

[Hoff]

You've presumably paid for a commercial product here and accordingly likely have at least some vendor support access available, and you've presumably copied over your (working) system disk images with BACKUP /IMAGE or related typical mechanisms, and you're presumably also following the vendor-provided configuration documentation, and unfortunately the results of this investment are less than satisfactory.  

At the risk of stating the obvious, isn't it time to call the vendor?

Emulators are more complex and more costly to install and to manage for the end-customer, are typically somewhat more fragile (taller app-stacks are like that), and various open-source and commercial emulators have had a history of occasional issues with their documentation, with limits or issues within their virtual networking implementations, and occasionally switch-level confusion with the connections into the network. 

In short: TANSTAAFL

The vendor is going to be most familiar with the emulator implementation, and with troubleshooting these sorts of problems.

Have you engaged the vendor support?

[Volker Halle]

Russ,

if you have another working OpenVMS cluster in that 'vault', enable LAT on those maschines and check whether the nodes in that cluster see their LAT service announcements - they should, if DECnet is working between them.

And then if your CHARON-VAXes don't see those LAT service announcements, after you've enabled LAT and outgoing LAT connections, you know for SURE, that you have a problem with LAN connectivity.

On the CHARON-VAX PC run NCC and check whether any packets are beign sent/received. Also on the OpenVMS VAX nodes, do **bleep**/SYS and SDA> SHOW LAN/COUNTER and check the send and receive counters.

Volker.

[Russ Carraro]

Volker,

The system disk was copied the production VAX.  Since the Charon VAX was given a new node name and address we thought it best to run netconfig on the Charon VAX.  From the Alpha (node LYFMS3), not a Charon Alpha which is in the vault,

LYFMS3> mcr ncp sho node 26.722

Node Volatile Summary as of 19-APR-2012 10:10:00

    Node           State      Active  Delay   Circuit     Next node
                              Links

26.722 (ATOQA1)                               EWA-1          0

and

LYFMS3> mcr latcp show   service

Service Name      Status       Identification
----------------  -----------  -------------------------------------------------
ATOQA1            Unavailable  @SYS$MANAGER:SIGNON.TXT

CARS              Available    LYNN SHOP FLOOR CLUSTER AXP ALPH
LYFMS1            Available    LYNN SHOP FLOOR CLUSTER AXP ALPH
LYFMS2            Available    LYNN SHOP FLOOR CLUSTER AXP ALPH
LYFMS3            Available    LYNN SHOP FLOOR CLUSTER AXP ALPH
MMCS              Available    LYNN SHOP FLOOR CLUSTER AXP ALPH
SFC               Available    LYNN SHOP FLOOR CLUSTER AXP ALPH

from the Charon VAX:

ATOQA1$ mcr ncp sho node 26.722

Node Volatile Summary as of 19-APR-2012 10:10:02

Executor node = 26.722 (ATOQA1)

State                    = on
Identification           = DECnet for OpenVMS VAX V6.1

and

ATOQA1$ mcr latcp show service

Service Name      Status       Identification
----------------  -----------  -------------------------------------------------
ATOQA1            Available    @SYS$MANAGER:SIGNON.TXT

I'm new to Charon stuff.  The customer has a vendor who does all the Charon configuring, patching, etc but I can log onto the Windows server and so I may be able to look at Charon configuration.  I can stop/start the service.  So I's be more than happy to look at something if you can tell me how to do it.

Russ

[Volker Halle]

Russ,

did you note, that the Alpha node LYFMS3 had AT LEAST ONCE seen a LAT multicast service announcement from your CHARON-VAX ATOQA1 ! So it must have worked at least once. Then maybe someone has tried at SET HOST/LAT ATOQA1 from LYFMS3 and it failed, so the service is now set to Unavailable !

This may be an indication, that only multicast or broadcast messages are sent/received from your CHARON-VAX. SDA> SHOW LAN/COUNT on the CHARON-VAX wil confirm this (check for PDUs received and Mcast PDUs received). If both have the SAME values, only Multicast/Broadcast traffic is reaching your OpenVMS VAX system. Then you have a CHARON-VAX network configuration problem and you need to run NCC from the CHARON-VAX utilities menue.

Volker.

[Russ Carraro]

lol at **bleep**/sys. is this the count you you asked about
LAN Data Structures
-------------------
-- EXA1 60-03 (DECNET) Counters Information 19-APR-2012 10:33:01 --

Last receive None Last transmit 19-APR 10:32:48
Octets received 0 Octets sent 6450
PDUs received 0 PDUs sent 129
Mcast octets received 0 Mcast octets sent 6450
Mcast PDUs received 0 Mcast PDUs sent 129
Unavail user buffer 0 Last start attempt None
Last start done 19-APR 10:00:48 Last start failed None

[Volker Halle]

Russ,

ANALYZE/SYSTEM then SDA> SHOW LAN/COUNTER and show me the first page...

Your data - just for DECnet Phase IV - shows, that this node NEVER received any DECnet packets at all.

Volker.

[Russ Carraro]

Volker

Yes I did try set h/lat atoqa1 from the LYFMS3. From ATOQA1 the ,PDs and Mcast PDUs are 0 received but have values for sent.

[Russ Carraro]

Volker,

SDA> show lan /count
LAN Data Structures
-------------------
-- EXA Counters Information 19-APR-2012 10:49:20 --

Seconds since zeroed 2913 Station failures 0
Octets received 0 Octets sent 0
PDUs received 0 PDUs sent 0
Mcast octets received 0 Mcast octets sent 0
Mcast PDUs received 0 Mcast PDUs sent 0
Unrec indiv dest PDUs 0 PDUs sent, deferred 0
Unrec mcast dest PDUs 0 PDUs sent, one coll 0
Data overruns 0 PDUs sent, mul coll 0
Unavail station buffs 0 Excessive collisions 0
Unavail user buffers 0 Carrier check failure 0
Frame check errors 0 Short circuit failure 0
Alignment errors 0 Open circuit failure 0
Frames too long 0 Transmits too long 0
Rcv data length error 0 Late collisions 0
802E PDUs received 0 Coll detect chk fail 0
802 PDUs received 0 Send data length err 0
Eth PDUs received 0 Frame size errors 0

Press RETURN for more.


Also, I found and am into NCC on Charon, if you want me to look at something.

Russ

[Volker Halle]

Russ,

so you definitely have a problem with LAN connectivity of your CHARON-VAX systems. No LAN packet ever reaches the OpenVMS VAX system running on top of CHARON-VAX.

Run the NCC tool = CHARON Network Control Center -> Monitor CHARON NIC activity and check, whether there are any packets being received or transmitted by the LAN adapter configured for CHARON-VAX to use.

Does NCC show any packets being sent or recevied ? Which operating system are you runnign CHARON-VAX on ? Did this EVER work after the CHARON-VAX installation and configuration ?

Did you reboot between the last 2 counter outputs ? Or why are all the EXA0 counters 0 whereas the EXA3 send counter are non-zero ?!

Volker.

[Russ Carraro]

Volker,

The VAX could have been booted, last time was around 10:00. I'm going to contact the Charon support person and have him look at the NIC configurations. I'll let you know what happens. Thanks for your help.

Russ

[Russ Carraro]

Volker,

Just wanted to keep you updated.  The Charon support guy hasn't gotten back to me. 

The Chaorn VAX version is 4.1.13402.  I have attached a document with screen captures of what I saw in NCC.

Russ