Operating System - OpenVMS
1748169 Members
4025 Online
108758 Solutions
New Discussion юеВ

Re: Varing response times for FTP file transfer process

 
SOLVED
Go to solution
Jon Pinkley
Honored Contributor

Re: Varing response times for FTP file transfer process

If I would have paid more attention to the original message I would have seen

"LANCP> sh dev/char"

So it is obvious that LANCP was available on the system John was using.
it depends
Phil.Howell
Honored Contributor

Re: Varing response times for FTP file transfer process

I would set up a test using ttcp.exe
this should show up any misconfiguration and you can then record the throughput for various transmission sizes
Phil
Wim Van den Wyngaert
Honored Contributor

Re: Varing response times for FTP file transfer process

Why would that affect large transfers more than small ones?

Memories. Very small files pass well and larger files don't pass at all. Just did some tests with a forced nofull while switch is expecting full.

Small file 8 block file : 3.7 MB per sec. 101 blocks : 2.5 MB/s
1469 blocks file : 0.38 KB/s.

Something caused by the number of collisions ? A back-off strategy ?

Wim
Wim
Wim Van den Wyngaert
Honored Contributor

Re: Varing response times for FTP file transfer process

Took a tcptrace and found that retransmission delay increases from a second to about 30 seconds (test of about 14 min in total). But with a randomizer picking values up to the maximum.

Wim
Wim
Jess Goodman
Esteemed Contributor

Re: Varing response times for FTP file transfer process

Jon Pinkley,

I lost your email address. Please send it to me at my last name at accuweather.com.
I want to send you a new tool I wrote that I think you will find very usefull in your environent.

(Why is there no way in these forums to send a private message to another member?)

Jess Goodman
I have one, but it's personal.
John A.  Beard
Regular Advisor

Re: Varing response times for FTP file transfer process

Once again, thanks to everyone for the replies.

I need to clear a few things up before I open myself up to further responses, so please excuse the additional text placed here. I am based in Canada assisting in the support of VMS nodes located in North America. The issue at hand is occuring on legacy servers located in Europe. Owing to a number of factors (age, pending retirement, etc), the 'local' support for these boxes has virtually disappeared. Myself and a few others here offer 'best effort' support when things go amiss across the pond, but we have no European VMS counterparts to communicate with. I also have to deal with a slight language barrier when I communicate with any on-site people, as the location of the boxes ranges from Austria, Poland, Germany thought to France. I can request that certain information (network stuff) be channeled back to me, but it is an extremely slow process to follow... hence delays in responding to your questions. I am also dealing with a 6 hour time difference. We have no access to any documentation on these boxes, and they have pretty well survivied (14 odd years) on their original configurations up to this point. As of this point in time, I do not have any specific information about the switch or the pipe connecting the various locations.

Back to the main issue. There are a total of three servers located in different countries connecting through a private network. One machine, MIAXP1, receives daily export files that are zipped before being FTD's to it. The source servers are AULIMS (the problem location) and EFVAXB (everything appears pretty normal).

The EFVAXB file is about 100MB and takes approximately 15 minutes to transfer. I cannot quantify at this moment (Europe gone for the day) how long the transfer from AULIMS normally took, but the 34MB job was still running after several hours this morning before the operator cancelled the run.

As for some of your recommendations, I used Jon's procedure to create 5,000 (+ 10,000) block files. the 5K file took about 90 seconds to create and the 10K one took about 130 seconds. I tried to reverse the FTP process and copied the resultant 10K file to AULIMS, and it took 80 minutes to complete.

I am going to post twice here, as it appears I cannot attach more than one file at a time.
I have included some brief explanations at the begining of each file, though I'm sure you will probably want to obtain more info along the way.

I am on a course tomorrow (Wednesday), so there will be a slight delay before I can get back to you again. I have asked the operator in Europe to see if he can take the file from AULIMS and FTP it to a different server altogether. There are several simialar VMS servers that also have files being FTP'd all over the place.

Thanks again for your patience and assistance... John
Glacann fear cr├нonna comhairle.
John A.  Beard
Regular Advisor

Re: Varing response times for FTP file transfer process

This is the output associated with the file creation process recommended by Jon
Glacann fear cr├нonna comhairle.
John A.  Beard
Regular Advisor

Re: Varing response times for FTP file transfer process

The attached is the zipped export file from EFVAXB .. the one that only took about 15 minutes.
Glacann fear cr├нonna comhairle.
Jon Pinkley
Honored Contributor

Re: Varing response times for FTP file transfer process

Since the problem seems to only apply to the transfers from AULIMS to MIAXP1 and not to transfers from EFVAXB to MIAXP1, it seems we can focus on AULIMS and its connection to MIAXP1.

Is the "private network" a shared network (like frame relay, MPLS, X.25) where the bandwidth is being shared with other entities? It does seem that something has changed in the link between AULIMS and MIAXP1, unless it has always been slow, or the amount of data has recently increased substantially. Does anyone have any access to reports of utilization of the link at the AULIMS site? Is it used for other systems besides the AULIMS server? If so, is there some other system at the site that is using it for a new use? Pending retirement of AULIMS suggests there is something that is replacing it. Is this new system doing "remote backups" or file synchronization over the same link?

Is this slowdown only affecting the AULIMS system? Does you company have any visibility into the "WAN" routers, or is this "owned" by the provider? Since you can't upgrade the VMS system, I am not aware of any tools you can use from UCX 4.1 (tcpdump isn't available as far as I remember, if it were you could at least capture a trace file to analyze with Wireshark).

If the link is overutilized, the answer is to buy more bandwidth. You are already compressing the data with zip, so any attempt to use a device to get additional throughput via compression isn't going to help with the zipped data.

Re:"Small file 8 block file : 3.7 MB per sec. 101 blocks : 2.5 MB/s 1469 blocks file : 0.38 KB/s. Something caused by the number of collisions ? A back-off strategy ?"

I was meaning 1300 blocks as small. I would expect a 1300 blocks file to cause "steady state" as far as ethernet timers are concerned, even the TCP connection should have passed the transitory effects, as long as there isn't some other traffic shaping in effect. Also, my experience with really small files is that the numbers reported are pretty bogus, as the VMS clock resolution becomes significant. (i.e. the ability to accurately measure the time for the transfer).

So it is definitely worth verifying that there isn't a duplex mismatch, but I will be surprised if that turns out to be the cause of the problem.

Cisco routers can be configured with congestion avoidance, a fancy name for dropping packets to cause TCP connections to back off before things melt down. Here's a pointer to a Cisco page about congestion avoidance, and how Cisco implements it. http://www.cisco.com/en/US/docs/ios/12_2/qos/configuration/guide/qcfconav_ps1835_TSD_Products_Configuration_Guide_Chapter.html Perhaps something like WRED or DWRED is in play, and affecting the large consumers more than the small consumers.

Jon
it depends
Wim Van den Wyngaert
Honored Contributor

Re: Varing response times for FTP file transfer process

John,

Could you do "ucx sho prot" before and after the slow ftp and post that ? (don't remember if ucx showed all protocols, so test the syntax first).

Wim
Wim