DECnet copy takes hours, FTP takes minutes

[Piet Timmers_1]

Hai everybody,

When I copy a file over DECnet it takes hours (even days), when I copy the same file using COPY/FTP it takes about 15 minutes.

Any idea where to look, we use DECnet over IP.

Greetings,

Piet Timmers

[John Abbott_2]

Hi Piet,

Some question:
1 What type of file are your trying to copy? 2 Size of file ?
3 What version of VMS ?
4 IP product + version ?
5 Are you sure that the IP ftp data xfer and the DECnet copy uses the same hardware NICs and routes through your network in the same way (hardware) ? We once had a situation where one connection was 100MB FD and another hopped into a 10HD segment.
6 check your counters for errors

You could trying using SET RMS/NETWORK_BLOCK_COUNT=bigger_no before issuing the decnet file copy.

Regards
John.

[Volker Halle]

Piet,

are you really sure, that the DECnet file copy is using DECnet-over-IP ? How do you address the destination node (nodename:: or node.domain:: or IP$a.b.c.d:: or xx.nn::).

A common symptom of 'slow' DECnet transfers would be lost packets. DECnet has a very long (in term of ethernet networks) timeout, before re-transmitting a packet. This would show as 'bad performance', while in reality it means long pauses before packet retransmission.

I normally use DTSEND to get an idea about the end-to-end throughput performance.

Volker.

[Volker Halle]

Piet,

I ran a simple DTSEND test on a local LAN between 2 DECnet Phase V nodes using NSP and DECnet-over-IP (had to toggle SESSION CONTROL TRANSPORT PRECEDENCE) and the differences in throughput for this simple test are minimal (within 10 percent):

DTSEND DECnet-over-IP 1.85 Mbit/sec
DTSEND DECnet-NSP 1.69 Mbit/sec

Just a simple DTSEND _test: DATA/NODE=xxx, no additional options.

Volker.

[Robert Gezelter]

Piet,

As has been noted previously, verify that the transmission paths are the same. The symptom that you are reporting is classic, often caused by incompatibly configured NICs (one full duplex, one half duplex -- the problem of lost packets only shows up under load).

Check all the error counters: hardware and software for BOTH IP and DECnet. Is the problem limited to these two nodes? Is there a router in the path between the two nodes with a configuration limit? Also, which DECnet over IP product are you using, and does it have any configurable settings?

- Bob Gezelter, http://www.rlgsc.com

[Kris Clippeleyr]

Piet,
If we encounter such phenomena, we always suspect the settings of the ethernet-card of the system and the settings of the port of the swithc to which the system is connected. We always make sure that the speed (10,100,1000) and the mode (full/half duplex) of the NIC of the system matches exactly the settings of the switch. We never let our systems autonegotiate.
Hope this helps.
Regards,
Kris (aka Qkcl)

[Volker Halle]

Piet,

OpenVMS V8.2 includes the

%EWA0, Possible duplex mode mismatch condition detected

console message and associated errlog entries for this type of situation (possible auto-negotiation problems). Check with LANCP> SHOW DEV/INTERNAL_COUNTERS EWAx for the recent console messages sent by the LAN driver.

For more details, see chapter 3.4 in the OpenVMS V8.2 New Features manual.

Volker.

[John Abbott_2]

Volker, RE: DTSEND

Thanks for the pointer, I need do do some h/w & s/w changes and this tool will be very useful.

Thank you !
John.

[Volker Halle]

John,

here are some usage notes for DTSEND:

- the DTR DECnet object (or SESSION CONTROL APPLICATION) on the remote node must point to a valid user account. For DECnet-Plus use: MC NCL SET SESS CONT APPLIC DTR USER NAME = "username" (e.g MIRRO$SERVER)

- DTSEND can only use DECnet Phase IV (6 char) node names

- if you want DTSEND to switch between using DECnet-over-IP and DECnet-OSI or NSP, you have to provide a way for session control to select the desired transport protocol. The default session control transport precedence is {OSI,NSP}, which you may need to temporarily change to {NSP,OSI} to prevent DECnet-over-IP to be used by default.

- if you want to test real throughput, you need to increase the packet size /SIZE=xxxx

DTSEND is a ready-to-run tool to test your network performance on OpenVMS systems.

Volker.

[Piet Timmers_1]

When I start the copy and use $ MCR LANCP SHOW DEV /COUNTERS, I see a increase of "unrecognized multicast destination packets" every second on the sending node.

Can this help.

Greetings.

[Volker Halle]

Piet,

I would expect this counter to increase at the same rate, even if you don't start the copy.

Something on your LAN is sending multicast packets, which your OpenVMS system receives, but has no 'consumer' (protocol/application) to handle them.

Can we find out first, which protocol you're using for the copy ?

Volker.

[Ian Miller.]

DTSEND is documented in Problem Solving manual
http://h71000.www7.hp.com/doc/82final/decnetplus/prb_slv.pdf

Section 3.10

[John Abbott_2]

Volker, thanks for the extended information, all noted. (I need to replace our aging FDDI ring with a "suitable alternative", but it's not a priority yet). :-) The /size qualifier is essential for this,as is the stats !

Ian, thanks for the manual, I missed that one !

Kind Regards
John.

[Piet Timmers_1]

Thanks, that manual is super.

I found out that the DECnet phase-4 address was different from the V5 address. So after moving the node to another location, the V4 address was changed, including scssystemid, but they have forgotten to update the V5 address.
I have updated everything now.

But still it is to slow.

The strange thing is, using DTSEND, I can see that this slow connection is only to one node in the network. The other way around, from the other node to my node is very fast. From my node to every other node in the network is fast, no problem. The other way around, from every node in the network to my node, no problem, a fast connection. So why is this one connection, from my node to one other node slow, even extreemly slow. I still can not see what causes this.

Greetings,

Piet Timmers

[Volker Halle · Accepted solution]

Piet,

now let's concentrate on that 'slow' connection between your node and the other node.

To find out, if there have been lost packets (and therefore re-transmissions making the connection look 'slow'), you need to find and look at the NSAP counters for the network connection between the 2 nodes:

Again, this depends on which network protocol you're using to communicate with the other node. In DECnet Phase IV times (NCP), this was easy: MC NCP SHO NODE other_node COUNTERS

You could use the SYS$SYSTEM:NET$MGMT application (needs an X11 display) and look at Tasks -> Show Known Node Counters, this will list the Retransmitted PDUs in the 3rd column (for each node and transport protocol)

It's a little bit harder with NCL:

For NSP transport:

NCL> show nsp local nsap *
NCL> show nsp local nsap remote nsap all counter

For OSI transport

NCL> show osi transport local nsap *
NCL> show osi transport local nsap remote nsap all counter

The counters to look at are:

Retransmitted PDUs
Duplicate PDUs Received

If they increase, while your DTSEND test runs to the 'slow' node, you have found an explanation at least WHY the throughput seems to be slow. The next step would be to try to find out WHERE the packets get lost.

Volker.

[Piet Timmers_1]

Volker thanks.

I see the following:

At time: 9:38 9:42

PDU's sent 37995 40022
Retransmitted PDU's 6488 6867
Duplicate PDU's received 1554 1650

Greetings.

Sorry: my formatting was good, the forum software does not understand.

[Volker Halle]

Piet,

o.k. so there ARE lots of PDU re-transmissions on both ends. Duplicate PDUs received typically indicate, that the ACK got lost.

2027 PDUs transmitted
379 PDUs re-transmitted - about 18% !!
96 Duplicate PDUs received

For every re-transmission, DTSEND will have to wait for a couple of seconds for the packet to reach the remote node. This make 'performance' to look quite 'slow'.

As you've now figured out how to look at the 'node counters', just repeat this test for a good node and verify, that there will be no PDU re-transmissions.

You then need a physical network plan and work out the route of the packets from your node to the remote node.

Let me ask again: are you really using DECnet-over-IP for the connection to that node ?

Volker.

[Piet Timmers_1]

Ok, just to be sure.

What can I do to let you know how Decnet is setup.

Piet

[Volker Halle]

Piet,

an easy way to verify whether you're using DECnet-over-IP is this:

While DTSEND is running to the remote node, issue a TCPIP SHOW DEV/PORT=399 (also try /PORT=102) on the remote node. If you're using DECnet-over-IP, you should see a BG device with the IP address of your local node.

With NCL you need to look at the session control port associated with the remote DTR session control application on the remote node:

NCL SHOW SESS CONT PORT * ALL

If you can find the transport port (what is it: OSI or NSP ?), try

NCL SHOW xxx TRANSPORT PORT ALL

What does Network Port show ?

Use can use /SECONDS=nn in DTSEND to keep the connection a little longer than 30 seconds while trying to look around with NCL.

Volker.

[Piet Timmers_1]

The transport is OSI.

Piet

[Volker Halle]

Piet,

that's what I had expected based on the description of 'slow performance'.

If you want to, we can continue this offline via mail (see my ITRC profile).

You now need to get a complete network diagram with all active components between your node and the other node. If you can identify another 'working' node near the 'bad' one, this would also help.

Are any DECnet routers involved (MC NCL SHOW ROUT CIRC xxx ADJ *) ? Or is it just an extended LAN ?

Volker.

[Volker Halle]

Piet,


The transport is OSI.


This can still be DECnet-over-IP. Did you check the TCPIP BG device ? Only the TRANSPORT PORT xxx Network Port information will tell the true protocol. For DECnet-over-IP it will show:

Network Service = RFC1006
Network Port = Generic Client User Process TPCONS VCM

Volker.

[Art Wiens]

Not sure about others, but I would encourage you to keep going "online". Volker this is a great tutorial!

Cheers,
Art

[comarow]

I'm assuming you are using the same path for both protocols. You might well have problems with your router.

However, just to test your local system,
you might copy to your own node,
$copy bigfile - mynode"usernamepassword"::disk:[directory]

to test your DECNET software.

Have fun.

[Volker Halle]

DTSEND is just fine for testing DECnet-related throughput performance problems, especially as it does not involve any disk IOs and does not consume any disk space...

Volker.