topic Re: LAT communication over WAN in Operating System - OpenVMS

LAT communication over WAN

Geoff Hess — Mon, 20 Oct 2008 14:49:35 GMT

I have a VAX/VMS system running VMS V6.2 and DECnet Phase IV. The VAX is performing a disaster recovery role for a production VMS system. There is currently DECnet and TCP/IP connectivity between the two systems over a WAN. I'm trying to establishing LAT communications over the WAN between the DR VAX and LAT terminals servers located at the site of the production VMS system. TSM output on the DR VAX shows a status of Unavailable for the terminal servers. (The terminal servers have been configured properly via DSV$CONFIGURE.COM.)

The attachment shows TSM output on the DR VAX followed by TSM output on the production VAX.

Is there anything that I might be missing or anything that I can check to help diagnose the problem? My network engineering contact tells my that the Cisco router on the DR VAX end should be tunneling the LAT packets between the DR VAX and the remote LAN, but I'm not 100% sure that this is accurate. Any help would be greatly appreciated!

Re: LAT communication over WAN

Jess Goodman — Mon, 20 Oct 2008 15:06:24 GMT

Sorry, but the LAT protocol is not routeable so it can not be used over a WAN unless you set up a VPN for it.

Re: LAT communication over WAN

Robert Gezelter — Mon, 20 Oct 2008 15:10:06 GMT

Geoff,

My solution would be to use an analyzer (e.g., WireShark) to check the actual transmissions between the systems.

That said, LAT over Wide Area Network connections is somewhat problematical because of issues related to propagation delay and the LAT protocol.

I have seen many situations with various protocols where misconfigured networks and routers have failed to forward packets in one direction or the other.

- Bob Gezelter, http://www.rlgsc.com

Re: LAT communication over WAN

EdgarZamora_1 — Mon, 20 Oct 2008 15:15:19 GMT

Whenever I have this problem I usually ask the network guy if the IP helper is configured properly on the router (between the two addresses).

Re: LAT communication over WAN

Geoff Hess — Mon, 20 Oct 2008 15:26:22 GMT

Guys, need to understand this better...
Several years back at the production site our VMS production system was communicating with DECserver 200s located across town via a T1 line. So first question: We must have been bridging LAT through our Cisco routers right?

Second question: Please explain the propagation delay issue. This is of concern because the DR VAX would be communicating with a PLC in which the normal production request/response time is < 320 milli-seconds. The production VAX has no problem maintaining reliable communcations, but I do need to determine how fast the remotely located DR VAX can communicate with this same device.

Re: LAT communication over WAN

Robert Gezelter — Mon, 20 Oct 2008 16:27:41 GMT

Geoff,

The LAT protocol has some timeouts that are reasonable in a LAN environment, but not necessarily workable in a WAN environment, particularly when there are any line errors. I do not have the citation handy, but it was a fairly well known fact when LAT was more heavily used.

Today, I would be tempted to use a telnet connection to the host in place of a LAT connection where there is potential for wide area access,

These problems would show up as dropped connections, not failure to see the services. Those are most likely caused by packets not reaching the receiver node.

- Bob Gezelter, http://www.rlgsc.com

Re: LAT communication over WAN

Steven Schweda — Mon, 20 Oct 2008 19:12:37 GMT

> We must have been bridging LAT through our
> Cisco routers right?

Right. For LAT, bridging works, routing
doesn't.

> Second question: Please explain the
> propagation delay issue.

I don't remember the fine print (probably
because I never knew it), but LAT has some
sensitivity to latency. That said, once,
upon a time long ago, I was at one end of a
9600b/s leased line between CA and MN with a
(forgotten brand, model) bridge+router at
each end. We never relied on LAT working
over that link, but it always worked when I
tried it (just fooling around, no serious
testing).

Re: LAT communication over WAN

Bill Hall — Mon, 20 Oct 2008 19:34:36 GMT

Geoff,

TSM uses MOP, not LAT, to communicate with your terminal servers. MOP (Maintenence Operations Protocol) is a non-routable protocol and has to be bridged or tunneled across a WAN link.

MOP uses three packet types, one for loopback testing, one for dump and load functions and one for remote console. The packet tyes to be bridged are 90-01, 60-01 and 60-02 respectively.

If you are using older DECservers that do not contain flash memory to load their software, you had better bridge MOP or get additional load hosts installed at your primary site. Otherwise if the VAXes are down at the primary site and the terminal servers are rebooted or loose power for what ever reason, they'll never get a down-line load. Your DR plan has a hole in it.

Bill

Re: LAT communication over WAN

Geoff Hess — Mon, 20 Oct 2008 20:54:26 GMT

>>Right. For LAT, bridging works, routing
>>doesn't.

>>TSM uses MOP, not LAT, to communicate with your terminal servers.
>>MOP (Maintenence >>Operations Protocol) is a non-routable >>protocol and
>>has to be bridged or tunneled across a WAN link.

So, IF LAT bridging (or possibly tunneling) is set up correctly, how do I verify that it's working, if not through TSM. It sounds like MOP also needs to be bridged or tunneled.

>>If you are using older DECservers that do not contain flash memory to
>>load their software, you had better bridge MOP or get additional
>>load hosts installed at your primary site. Otherwise if the VAXes are
>>down at the primary site and the terminal servers are rebooted or loose
>>power for what ever reason, they'll never get a down-line load.
>>Your DR plan has a hole in it.

Good catch. The original plan has been to relocate the DR VAX to the LAN where the "event" has taken place. (If a LAN still exists of course...) The DR VAX has the terminal server software on it and can function as a load host. Only recently was I asked to test LAT across the WAN.

Re: LAT communication over WAN

Bill Hall — Mon, 20 Oct 2008 21:38:58 GMT

Geoff,

$MCR LATCP SHOW SERVICE will show you the LAT service messages that the DR system can "hear" on the network. If you see your production server's LAT service you should be able to $SET HOST/LAT if outgoing connection are enabled.

"Outgoing connections" have to be enabled in LATCP. Outgoing connections can be enabled by $MCR LATCP SET NODE /CONNECTIONS=BOTH (both means incoming connections, the default and outgoing connections). This has to be enabled at every startup.

Bill

Re: LAT communication over WAN

Colin Butcher — Tue, 21 Oct 2008 06:22:18 GMT

Don't forget to bridge the "remote console carrier" protocol as well as the MOP protocol. MOP is the down-line load mechanism (which you will need for flashRAM updates even if you boot your DECserver locally from flashRAM). "remote console carrier" is how you reach the network console of the DECserver, for example "ncp connect node ", or indeed TSM operations. MOP and "remote console carier" work in co-operation when working with DECservers for, but they have different purposes and different protocol type values.

In summary:

- bridge LAT, MOP and "remote console carrier". See RFC1700 or the DECserver documentation for the protocol type values etc. (60-01 = MOP load, 60-02 = remote console carrier, 60-04 = LAT).
- get DECservers with flashRAM for local boot and make sure you load the 'right' version of firmware into them ("decserver_prompt> init from ethernet update flash delay 0"). Make sure you get DECservers with big enough (>= 2MB) flashRAM to boot an uncompressed load image, otherwise you'll be there for a while waiting until you can use it. DOn't boot remotely across a WAN link!
- Use the output on port 1 of the DECserver to watch the DECserver boot process (unless you've defined the DECserver console to a different port).
- check the round trip delays on the network. Simple timing using DECnet DTSEND or writing a piece of code yourself is probably a good start. If they look poor (ie bad latency) then think about changing some of the LAT timer values.
- Consider moving from LAT / MOP based DECserver usage to TCP/IP based DECserver usage, especially if you're booting locally from flashRAM. You'll need to enable TELNET LISTENER at minimum to get to the network console port (on port 23 by default). However, this might be a step too far right now as it involves making changes to the way your DECservers are set up rather than just getting it all to work "as is". Depends which DECservers you have - not all support TELNET as well as LAT. 900TM and 90M+ are fine though.

Cheers, Colin (http://www.xdelta.co.uk).

Re: LAT communication over WAN

Wim Van den Wyngaert — Tue, 21 Oct 2008 06:30:05 GMT

Which mop do you use (ncp, ncl, lancp) ?
Are any *mop* processes active ?

Are the decservers defined as mop clients ? Could be of importance if "only known clients" is set to true.

And how far are the decservers located from the vms nodes ? We had no problems with lat worldwide in the 90's (unkown infra).

Wim

Re: LAT communication over WAN

Geoff Hess — Tue, 21 Oct 2008 15:24:14 GMT

Bill,

>>$MCR LATCP SHOW SERVICE will show you the LAT service messages that the DR system can "hear" on the network.

The only service that I have defined on the DR VAX is itself (nodename DCQUIK), so when I do a MCR LATCP SHOW SERVICE it shows only DCQUIK as a service. No other services are reported and not unexpectedly, doing a SET HOST/LAT service-name to the production system (service-name ENVAXA) results in the error: %LAT-F-NOSVC, service name ENVAXA not found. If LAT bridging is working, should I be seeing the remote LAT services via LATCP?

If not necessarily, then I'm still without any obvious way of determining whether LAT bridging is functioning.

p.s. Colin, thanks for the valuable info on the MOP and remote console carrier protocols.

Re: LAT communication over WAN

Andy Bustamante — Tue, 21 Oct 2008 15:30:26 GMT

Geoff,

>>>Several years back at the production site our VMS production system was communicating with DECserver 200s located across town via a T1 line. So first question: We must have been bridging LAT through our Cisco routers right?

It depends. Cisco has support to translate between TCP/IP and LAT. With the correct Cisco licensing, you can translate a LAT service into TCP/IP, carry the traffic over a WAN and return to LAT on the remote side. Your networking team can answer this. In recent years it's become trendy to recognize TCP/IP as the only legitmate network traffic, and it's not unusual to drop functionality during network upgrades.

Andy

Re: LAT communication over WAN

Hoff — Tue, 21 Oct 2008 15:50:56 GMT

Your LAN bridge or VLAN bridge is not passing LAT protocols.

It appears that your networking folks don't have non-IP bridging set up correctly.

When dealing with non-IP protocols and "modern" managed switches, I've learned to always verify what the networking folks are telling me about the configuration. Trust, but verify.

This can include creating and shipping a raw 60-04 packet to a host on the same subnet as the target host. But here, you've already got good evidence with what you're seeing, though you seem to be putting more weight in what you're being told than what you're seeing. (Rather than tossing your own 60-04 or such, Wireshark or another packet sniffer would be another approach, and there are various LAT monitors within various network devices and within Freeware packages such as DBS-LATWATCH or the monlatv tools.

http://mvb.saic.com/freeware/freewarev50/dbs-latwatch/

http://mvb.saic.com/freeware/vax89a2/monlatv/

You do need to discuss this your networking folks. (Not really with us; we can provide moral support and information, but not a resolution.) (Shortly after receiving the reports you are making to your networking team here, I'd have expected the team would have fired up a protocol monitor akin to Wireshark, and looked for the LAT traffic.)

Now if you'd like to bring in some outside help to prove to your network folks that the LAN or VLAN is set up incorrectly (and to take political heat, etc), that's another matter.

Re: LAT communication over WAN

Geoff Hess — Tue, 21 Oct 2008 16:58:25 GMT

Hoff,

I suspected that the LAT bridging was not functioning, but I wanted to go back to the network people with something that would validate this. I haven't recognized anything that I've done and shared up to this point that positively confirms that the network is not configured correctly to handle the LAT messages. Since TSM uses MOP not LAT, apparently terminal server unavailability shown there doesn't prove it. Does the absence of remote services showing in LATCP prove it?

I will be asking the network team to confirm that it's working. Thank you and everyone else for the valuable suggestions.

Re: LAT communication over WAN

Bill Hall — Tue, 21 Oct 2008 18:20:22 GMT

Geoff,

>>Does the absence of remote services showing in LATCP prove it?

Yes, that is proof enough without putting a packet sniffer(s) on the network. LAT uses the multicast service announcements to get the remote service's address.

BIll

Re: LAT communication over WAN

Geoff Hess — Tue, 21 Oct 2008 19:51:53 GMT

Revisited Bill Hall's comments and enabled outgoing connections:
MCR LATCP SET NODE/CONNECTIONS=BOTH

Shortly after that was done, service nodes started to appear when doing a MCR LATCP SHOW SERVICE command. I can now do a SET HOST/LAT service-node. I can successfully SET HOST to the systems in the same building as the local system. The remaining systems are geographically remote and are timing out (LAT-F-TIMEOUT error), regardless of the value of the LAT retransmit value. But at least, LAT connectivity appears to be there.

Many thanks to all of you for the education and help to allow me to get this far.

Re: LAT communication over WAN

Allan Bowman — Wed, 22 Oct 2008 23:27:18 GMT

I just wanted to add another possible approach to consider in the future. We had a similar setup - VMS 6.2, DECnet IV, and DECserver 200's with WAN connections between multiple sites. We also had MultiNet TCP/IP running. Because of customer complications, we could not run DECnet on their WAN, so we used DECnet over IP (using MultiNet). We did not need direct LAT connectivity to our DECservers over the WAN because of separate MUXed circuits for the ports, but over time we had DECservers die and replaced them with DECserver 300's. The 300's do support TCP/IP, and using MultiNet we were able to connect to a remote DECserver even if the remote VAX was down (and we eliminated the need for our MUXes and circuits).

I realize this isn't your exact situation, but you may want to consider converting to the 300's for more flexability moving forward. I'm not sure how many are available out there, but I do have access to at least 10 of them (my former employer is in the process of liquidating).

Allan in Atlanta