topic Re: Very slow connections in Operating System - OpenVMS

Very slow connections

Antoniov. — Thu, 27 Jan 2005 03:22:47 GMT

My customer has an old alphaserver 400 with openvms V6.2, Advanced Server V6.0, UCX V4.2
This system has been slowing for three weeks and I'm becoming insane because of this.
The axp is a remote site without direct access and it's uneasy making any type of check.
The information about this trouble are:
1) All terminal connections are via LAT or telnet and are very slow; sometime users wait for omly echo of characters.
2) The only direct connection without lan is from console and it works always quick.
3) Shared directory of pathwork are slow too.
I guess it's a lan trouble but I can't solve it.
Any clue, hint, suggestion will be appreciated.

Antonio Vigliotti

Re: Very slow connections

Bojan Nemec — Thu, 27 Jan 2005 04:03:18 GMT

Antonio,

As you say it seems a network trouble. Probably hardware. Have you try to look at the lan card counters (mc lancp show device /counters)? Are there any errors?

I am shure that you have checked the card/switch speed and mode settings.

How is the lan configurated (switches,hubs)? If you have a managed switch, look at the port statistics. Sometimes a broken PC lan card can congest the network. Can you isolate some parts of the network, and see if it works better without them?

Sory to respond to your question with more questions.

Bojan

Re: Very slow connections

Daniel Fernandez Illan — Thu, 27 Jan 2005 04:03:56 GMT

Antoniov
Are you checked DECnet address of clients?.
It's possible that a client has the same address of Server?.
Other posibility is check phisicals connections of LAN (packets lost...)
Saludos "Desde un lugar de la Mancha" (400 years ago).
Daniel

Re: Very slow connections

Bojan Nemec — Thu, 27 Jan 2005 04:53:26 GMT

Antonio,

As Daniel suggested, check also for duplicate IP addresses.

Bojan

Re: Very slow connections

Uwe Zessin — Thu, 27 Jan 2005 04:57:11 GMT

Yes, looks like a bad network to me, too. Perhaps somebody has screwed up the termination if this is thick- or thinwire cabeling.

I'd check with
$ MCR NCP SHOW KNOWN LINE COUNTERS
or
$ MCR LANCP SHOW DEVICE /COUNTERS

Re: Very slow connections

Antoniov. — Thu, 27 Jan 2005 05:03:06 GMT

Bojan,
because the system is far out I can do only limitated checks.
I found many errors in LATCP service. Customer network of Pcs works fine. Changed port of hub-switch, didn't resolve. Set ewa0 full duplex and/or fast: didn't resolve.
Daniel,
the only decnet address is on AXP.

Antonio Vigliotti

Re: Very slow connections

Antoniov. — Thu, 27 Jan 2005 05:07:22 GMT

Uhm,
duplicate ip address mey be a good start point.
Uwe,
it's installed decnec IV conseguently I can't use LANCP commands. I looked at NCP counters but no errors found (like LAT); however decnet is not in use.

Antonio Vigliotti

Re: Very slow connections

Uwe Zessin — Thu, 27 Jan 2005 05:14:31 GMT

LANCP is not a DECnet component - one of its uses is to provide a MOP download _independent_ from DECnet (IV or V). I thought that it's device counters are basically the same as DECnet's LINE counters, because they come from the hardware interface.

Re: Very slow connections

Volker Halle — Thu, 27 Jan 2005 06:50:31 GMT

Antinio,

I found many errors in LATCP service

What are those errors ? Could you provide the output of the appropriate LATCP command ?

Volker.

Re: Very slow connections

Antoniov. — Thu, 27 Jan 2005 09:49:46 GMT

Volker,
the answer is no. Every time I have to do something I have to call my customer and connect my emulator to his system.
Now I'm waiting for my customer who is checking for duplicate ip address.

Antonio Vigliotti

Re: Very slow connections

Doug Phillips — Thu, 27 Jan 2005 18:03:06 GMT

Has a Win2k or 2k3 server been added to the network recently?

Re: Very slow connections

Garry Fruth — Thu, 27 Jan 2005 18:10:31 GMT

I would not expect a duplicate IP address to affect LAT performance. I think you may be in for a road trip.

Re: Very slow connections

Lawrence Czlapinski — Thu, 27 Jan 2005 19:29:15 GMT

Antonio,
I usually only see this problem on VAXen.
1. How much of the CPU is being used? Just to make sure the Alpha CPU(s) aren't saturated. If the CPU usage is in the 90's for an extended period and the CPU queue is long, the system may be saturated. If this is the case, you will need to investige furhter whether there may be a CPU bottleneck.
2. If you don't have a CPU bottleneck,
do you have excessive buffered or disk IO?
3. Do you have excessive pagefaulting or swapping?
4. If none of the above, then it's probably a LAN problem. Are the LAT and TELNET connects going over ethernet? If yes, then the network group needs to check the LAN. We had this sort of problem a couple times when some of our network PCs got hit by a worm that caused them to fill the bandwidth with ping requests. You will need to have the network group check the LAN traffic with a sniffer to see what is happening. I use an AVAIL_MAN account on 1 node to watch the CPU usage of our 21 nodes across 2 sites. I have it installed on a couple more nodes in case I need it.
5. Is the problem most of the time or occassionaly? When the processes are slow, there may be higher priority compute bound processes running.
6. Are the WSDEF and WSQUOTAs adequate for the processes that are running slow? If these are too low, you could be getting a lot of pagefaulting. This is more likely on a VAX than on an Alpha.
7. The processes may be loading a lot of DCL images. This can cause a lot pagefaulting.
Lawrence

Re: Very slow connections

Willem Grooters — Mon, 31 Jan 2005 06:23:07 GMT

My customer has an old alphaserver 400 with openvms V6.2, Advanced Server V6.0, UCX V4.2
This system has been slowing for three weeks and I'm becoming insane because of this.

My assumption is it has worked fine in the past. So something has changed three weeks ago - whatever/anywhere: OS, Network, hardware, or, as others already mentioned, there is some hardware trouble - NIC (on Alpha), Hub or switch.

Have you looked to the routing table on the Alpha? I recall I have seen a similar problem caused by multiple routes to a network over different gateways - one of these wrong. I don't know how it got there (was not explicilty added) but once I removed the wrong one, the problem was gone.

Re: Very slow connections

Willem Grooters — Mon, 31 Jan 2005 06:26:52 GMT

Antonio,

an addition: The routing problem can exist on routers in between as well. Route tracing may tell you more.

Willem

Re: Very slow connections

labadie_1 — Mon, 31 Jan 2005 08:50:25 GMT

can you post

mc ncp sh k line coun
mc ncp sh exec coun
mc ncp sh k node coun

it may reveal a few things

Re: Very slow connections

Antoniov. — Mon, 31 Jan 2005 09:28:37 GMT

Hello to everybody,
my customer told me all IP addresses of network: no duplicate address.
Three weeks ago it changed motherboard (and NIC too) with same model. Executed AUTOGEN with some little difference (e.g. MAXPROCESSCNT).
System is intermittently slow. Because my customer can't give me any important notice I'll go to him on this thursday and therefore will be able to analyze various counters.
I think do this:
- look at physical and logical interface.
- look for decnet, lat and ucx counters then reckon error.
- compare lan process (telnet) with direct process (serial port).
I'll bring a working NIC.

Antonio Vigliotti

Re: Very slow connections

Garry Fruth — Mon, 31 Jan 2005 11:38:36 GMT

If the machine can send e-mail via the internet. You could set up a command procedure that polls those counters on a daily or twice-daily basis and e-mail you the results. If e-mail is not available, you may want to record the counters to a log file.

Re: Very slow connections

Antoniov. — Wed, 02 Feb 2005 10:41:18 GMT

My customer delayed meeting in next week :-(
Connected to his system.
$ MC NCP SHOW EWA-0 COUNT
>65534 Seconds since last zeroed
195556 Data blocks received
63900 Multicast blocks received
309 Receive failure, including:
Block check error
Framing error
10106316 Bytes received
3922904 Multicast bytes received
0 Data overrun
140470 Data blocks sent
27955 Multicast blocks sent
195 Blocks sent, multiple collisions
86 Blocks sent, single collision
298 Blocks sent, initially deferred
29921484 Bytes sent
2945580 Multicast bytes sent
1 Send failure, including:
Excessive collisions
0 Collision detect check failure
0 Unrecognized frame destination
0 System buffer unavailable
0 User buffer unavailable
$ MC LANCP SHOW EWA0 /COUNT
Value Counter
----- -------
4294967295 Seconds since last zeroed
196201 Data blocks received
63939 Multicast blocks received
20250627 Receive failure
10136609 Bytes received
3925319 Multicast bytes received
0 Data overrun
140998 Data blocks sent
27970 Multicast packets transmitted
195 Blocks sent, multiple collisions
86 Blocks sent, single collision
298 Blocks sent, initially deferred
30057684 Bytes sent
2946745 Multicast bytes transmitted
65537 Send failure
0 Collision detect check failure
0 Unrecognized frame destination
0 System buffer unavailable
0 User buffer unavailable

Anotnio Vigliotti

Re: Very slow connections

Uwe Zessin — Wed, 02 Feb 2005 10:57:17 GMT

So it's not urgent for him, eh? Well, I, too, have such a project running at the moment - let's share our ":-("s

Apparently something at the hardware layer is corrupting frames and causing other errors.