- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Varing response times for FTP file transfer proces...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 08:21 AM
тАО06-09-2008 08:21 AM
I am noticing an anomaly in transfer times when it comes to pushing files (FTP) between two VMS servers across our European WAN.
Anything over 40,000 blocks is taking a very very long time to complete. When I tried to test things using files (smaller) of differing sizes, the job tended to complete in a much more timely manner.
When I specified a file with c. 1,300 blocks, it took less than 50 seconds. I then worked with a file of about 5,000 blocks, and this took about 9 minutes. It appears that the larger the file, the greater the problem becomes. Is there some issue with FTP and the actual size of the file being transferred?.
These are two old Alpha 1000s, and the version of TCP/ip is Digital TCP/IP Services for OpenVMS Alpha Version V4.1 - ECO 10. These, and many other similar European based servers are due for decommisioning within 12-18 months, so business does not want these boxes upgraded. The FTP file transfer process has been functioning without incident for many years. The source machine is currently down, but here is the interface spec of the target server.
LANCP> sh dev/char
Device Characteristics EWA0:
Value Characteristic
----- --------------
Normal Controller mode
External Internal loopback mode
CSMA/CD Communication medium
16 Minimum receive buffers
32 Maximum receive buffers
No Full duplex enable
No Full duplex operational
TwistedPair Line media
10 Line speed (megabits/second)
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 08:53 AM
тАО06-09-2008 08:53 AM
Re: Varing response times for FTP file transfer process
I'd recommend looking at the network with a network analyzer to get a better picture, and find out how the WAN is constructed (if a private WAN). It may be as simple as congestion, or as difficult as IP stack bugs. Some WANs have traffic shaping devices that will shift FTP traffic down to lower priority.
I can recommend a couple of things to try, though:
- How does 'pulling' the data (FTP get) compare to 'pushing' (FTP put) the data?
- How does this compare to FTP times for data between other operating systems at these endpoints?
- Ping the remote system during a transfer and see how the times compare to when there is little activity.
- If the stack supports it, the FTP hash command can provide some idea (when run interactively) of whether the entire transfer is slow or whether the slowness is intermittent.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 09:03 AM
тАО06-09-2008 09:03 AM
Re: Varing response times for FTP file transfer process
May be the route is not always the same, and sometimes you pass through a saturated router.
Can you do
$ @sys$startup:tcpip$define_commands
and then
$ traceroute ftp_node
for both nodes.
$ netstat -m
and
$ netstat -s
can show, for both nodes, interesting things.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 09:10 AM
тАО06-09-2008 09:10 AM
Re: Varing response times for FTP file transfer process
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 09:38 AM
тАО06-09-2008 09:38 AM
Solution- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 01:35 PM
тАО06-09-2008 01:35 PM
Re: Varing response times for FTP file transfer process
I concur with the attempt to transfer to NL: for the purpose of guaranteeing that the problem is not file extends. I would also suggest raising the buffering and blocking factors on the local end to ensure that disk delays and fragmentation have less of a [potential] impact.
Do you have any statistics for the links on both sides?
- Bob Gezelter, http://www.rlgsc.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 02:42 PM
тАО06-09-2008 02:42 PM
Re: Varing response times for FTP file transfer process
You indicate two hosts, but do discuss the presence of other hosts. Do these transfer times involve just these two hosts, or are multiple hosts and network paths involved?
You show a 10 megabit Ethernet controller, but provide no indication of the connection between the two sites. How big is the pipe? 10 MbE all the way?
Disk fragmentation is certainly a potential cause, as will be file creation settings in the context of the target system.
Here are some old tuning documents:
http://h30266.www3.hp.com/odl/vax/network/tcpipv42/manage/6526pro_018.html
Look to something akin to this in the target system login context or (as shown) system-wide:
$ DEFINE /SYSTEM /EXEC UCX$FTP_WNDSIZ 4096
$ DEFINE /SYSTEM /EXEC UCX$FTP_FILE_ALQ 500
$ DEFINE /SYSTEM /EXEC UCX$FTP_FILE_DEQ 500
FTP is not fast to start with, has issues when firewalls are involved, and tends to degrade badly in the presence of network errors or high-latency connections.
You can up the window size, but that doesn't usually help all that much.
Since you're not swapping anything, see what a multi-step process might give you over the same link. Try Mac OS X to Mac OS X or another local client, for instance, over the same IP link, pushing the same file size around. (And the obvious one-plus, if that transfer looks good -- see if zipping the file (with "-V") and tossing it over to the Mac (or whatever) and sending the file from there makes this transfer go faster.
You're on very old software and very slow hardware, and the no-upgrade policy basically trumps most discussions here. And if these are really AlphaServer 1000 boxes (and not 1000A boxes), do make sure nobody is using the graphics console when these FTP transfers are running.
Do you have DECnet-Plus installed on both ends? If so, DECnet over IP might be an option; another path and protocol to test.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 03:13 PM
тАО06-09-2008 03:13 PM
Re: Varing response times for FTP file transfer process
Since you are running TCP 4.1, what version of VMS is running (6.x?).
1300 blocks in 50 sec is 650K/50s = 13KB/sec ~ 104Kbps
5000 blocks in 540 sec is 2500K/540 = 4.6KB/sec ~ 37Kbps
Time to transmit 5000 blocks at 13KB = 2500KB/13KB/SEC = 192 seconds
While fragmentation on the receiving end, could contribute to the delays, if seems unlikely that it would contribute 346 seconds in a 5000 block file, since we are just adding to the end of the file, and worst case, we would have something like 70 extension file headers (assuming 1 block cluster size and the unlucky event that each file extension is only able to get 1 block at a time). More likely would be some other disk activity at the time of the transfer (my opinion).
How long does it take to create a 5000 block file using something like this on the receiving end?
$ open/write testing123 test.dat
$ cnt = 0
$ write sys$output "Starting at ''f$time())'"
$ rec = f$fao("!508*z") ! 508 "z"s, with 2 byte VFC + 2 byte rec length, this is 1 rec per block
$_loop:
$ write/symbol testing123 rec
$ cnt = cnt + 1
$ if cnt .lt. 5000 then goto _loop
$ close testing123
$ write sys$output "Done at ''f$time()'"
$ exit
If that takes 346 seconds (or even 200) to run, then fragmentation may be a significant factor.
Given only the limited info you have provided, and assuming that the only variable is the size of the transfer, I think that traffic shaping is a good possible explanation. The transfer of the data in FTP is in a single FTP connection, and it is reasonably easy for routers to give lower priority to "bulk" traffic like FTP. I don't think that doing a ping to the other end while the transfer is in progress will necessarily give you an accurate indication of the limiting that the FTP is seeing, since ping normally uses ICMP packets, and these would not be treated as part of the FTP data.
The only reliable way to see rate limiting is with a protocol analyzer, like the free Wireshark.
Traffic shaping is similar in concept to the "priority adjusting" programs that lower priority of CPU hogs. FTP tends to use as much bandwidth as possible, and the packets are generally at the MTU. Traffic shaping tries to give the light users a break by letting them go to the front of the line. You may be able to get better performance by buying a better class of service, or possibly by trying to make your traffic look like something else, (encapsulation) or reduce the size of the files (zip) and make it a bunch of shorter TCP connections (trying to defeat traffic shaping) by splitting the file into more small pieces (zip followed by zipsplit). Depending on the traffic shaping, this may or may not have any effect, as it may be limiting based on source/destination ip, or may know that the multiple data connections are related to the same ftp control connection.
You could try an FTP to the null device, but you should probably watch the data with Wireshark while you do that to see if the rate gets throttled back as the transfer progresses.
Jon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 10:13 PM
тАО06-09-2008 10:13 PM
Re: Varing response times for FTP file transfer process
As nothing changed on VMS, I would suspect the network changed. May be they replaced the switch and you should have duplex enabled now. mc lancp show dev/cou will show many collisions if that is the case.
Wim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-09-2008 10:51 PM
тАО06-09-2008 10:51 PM
Re: Varing response times for FTP file transfer process
Why would that affect large transfers more than small ones?
But that does bring up another possible test, can an FTP transfer be done between two system on the same LAN, that would tell you if the problem was in the WAN or the LAN or system (fragmentation).
I can't remeber: was LANCP available in 6.x?
Jon