Operating System - OpenVMS
1819787 Members
3466 Online
109607 Solutions
New Discussion юеВ

ftp large files fail, small files succeed

 
SOLVED
Go to solution
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

first, an update: they fixed the firewall, I can connect directly to hnatst from the vpn now...but it didn't fix/affect the ftp issue.


so no, that log file in sys$manager ends with a run of tcpip$ftp_child.exe. previous versions of that log file show that and then the logout message.

here's the tcptrace - I ran it on the test system first, it took so long that I was hesitant to run it on prod, not knowing what it's doing to the network:

HNATST> tcptrace/full/port=local=20/prot=tcp/packet=200
HNATST::_TNA15: 09:42:52 TCPIP$TRA CPU=00:00:00.31 PF=853 IO=471 MEM=303
0110FC0A B8160201 00006AA7 1C00C045 0000 E....j..........
00000000 00000000 9BEE6411 | 010000E0 0010 .....d..........
0000 00000000 00000000 00000000 0020 ..............
0110FC0A D9120201 000049AB 1C00C045 0000 E....I..........
00000000 00000000 9BEE6411 | 010000E0 0010 .....d..........
0000 00000000 00000000 00000000 0020 ..............
0110FC0A F80E0201 00002AAF 1C00C045 0000 E....*..........
00000000 00000000 9BEE6411 | 010000E0 0010 .....d..........
0000 00000000 00000000 00000000 0020 ..............

sorry for the formatting. this was still running after 8 minutes, I'm not sure when/if it's going to end...


Hoff
Honored Contributor

Re: ftp large files fail, small files succeed

> I didn't see any ftp logs, the [tcpip$ftp] directory has none. ftp is configured the same on both nodes.

I had asked if the ftp server was configured and running on the target box, not if it was "the same". If the two hosts are "the same" and "the ftp server is not configured" (which still looks to be in play here), then ftp connections headed the other way may well be broken, too.

That there are no logs in the ftp server directory and in the target user's default directory - the two spots where the logs are expected - implies that the ftp server is not running.

>can't get much simpler than that, I imagine!

Other than localhost, no. That's a simple network.

Given that there are managed switches here, there is a second whole layer of weird in play.

>on a side note, the previously requested ping with packet sizes worked like this:

FWIW, I'd be careful with tossing big pings around. That can crash some boxes, including certain versions of VMS. qv: Ping Of Death.

>> Post the full command sequence.

>by this, I assume you meant what I have included in the first post of this thread - were you looking for something more than that?

Yes, I was wondering about the connect command, and your configuration statement of this-to-that-to-other does make the network appear, um, uniquely configured.

I was also wondering why you were not using the COPY /FTP /BINARY command that's been around in DCL and IP since V6.2, but that's another discussion. Way easier to script that one into a DCL procedure, too. But I digress.

>> Post logs from the target ftp server.

>don't see any logs there either, in sys$sysdevice:[tcpip$ftp]

That's a potential problem. Do look into why you have no logs. A potential trigger here is that the ftp server is not configured and started. And if it's not obvious, the ftp server is different from the ftp client.


>> And if that's a full OpenVMS distro kit as it looks, burn a CD and send it over.

>not sure what you're looking for here, I don't have burners on these systems. they are an es45 and an alpha 4100, with 100mb nics.

OK, translating that apparent confusion, you don't know how to burn optical disk media for use with VMS.

Presuming your software licenses permit it, burning a disk to boot VMS is not particularly difficult, even if you have to haul the disk image over to a Windows or Mac OS X or Linux or Unix box to burn the disk.

At its simplest, you create a 700 MB LD disk partition, BACKUP /IMAGE the distro into it, dismount the LD and copy the backing file over to the box where you have a burner on (if you don't have a burner on VMS, and it's handy to have a burner on VMS even if it's comparatively slow) and burn the disk. Then transport the media. Even with the glacial burn speeds on VMS, I can have a disk generated in well less than a week.

I have directions posted for that elsewhere on the 'net.

http://labs.hoffmanlabs.com/node/28

>I did bring up the firewall issue earlier, but someone didn't think that mattered.

You're thinking of a "firewall" as a device at the network perimeter. That's certainly a common case, but not the only case.

A managed switch can be a very effective firewall.

And it's distinctly possible that various production boxes will have their own firewalls; that's less common with VMS, but it's near-ubiquitous configuration with other platforms (Windows, Mac, Linux, Unix), and it may well be a possibility with VMS guested on a HP-UX box.

This prevalence of firewalls is one of the central reasons why ftp is stinky. Well, that and ftp exposes your access credentials in cleartext.

As for sftp, you type "sftp" at the login prompt. TCP/IP Services versions V5.4 and later have sftp available.

I have various postings on sftp up, including setting up a secure certificate-based, no-password login.

http://labs.hoffmanlabs.com/node/1118

I'd suggest getting an escalation process on-line, too, and ITRC isn't a particularly good one. (I've done hospital IT, and know from experience that the ER docs can get all the way from zero to full cranky milliseconds after the servers or the network have gone walkabout.)
Oswald Knoppers_1
Valued Contributor

Re: ftp large files fail, small files succeed

>HNATST> tcptrace/full/port=local=20/prot=tcp/packet=200

No no, please do this on HNAA, not on HNATST. Or change the command to /port=remote=20

Oswald
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

quick answer is if ftp server wasn't running, then I wouldn't be able to ftp a small file either.......unless I'm missing something?
longer answer is that ftp under "server options" on both systems is started and enabled.

as you can see below, the log file is configured, I don't know why it's not coming out...

HNAA> tcpip sho serv ftp/full

Service: FTP
State: Enabled
Port: 21 Protocol: TCP Address: 0.0.0.0
Inactivity: 5 User_name: TCPIP$FTP Process: TCPIP$FTP
Limit: 10 Active: 1 Peak: 2

File: TCPIP$SYSTEM:TCPIP$FTP_RUN.COM
Flags: None

Socket Opts: Rcheck Scheck
Receive: 0 Send: 0

Log Opts: Acpt Actv Dactv Conn Error Exit Logi Logo Mdfy Rjct TimO Addr
File: SYS$SYSDEVICE:[TCPIP$FTP]TCPIP$FTP_RUN.LOG

Security
Reject msg: not defined
Accept host: 0.0.0.0
Accept netw: 0.0.0.0
HNAA> dir sys$sysdevice:[tcpip$ftp]

Directory SYS$SYSDEVICE:[TCPIP$FTP]

LOGIN.COM;1 TCPIP$FTP_ANONYMOUS.LOG;1

Total of 2 files.
HNAA>



thanks for the note on big pings...gulp!

disregarding the security issues, I don't see how copy/ftp/binary (which I know about) or sftp would work if ftp doesn't - am I missing something?...sftp does exist but it's not configured.

by the way, I aborted the tcptrace after 20 minutes...not sure how long that was supposed to run or what it was supposed to tell me.
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

>>HNATST> >tcptrace/full/port=local=20/prot=tcp/packet=200

>No no, please do this on HNAA, not on HNATST. >Or change the command to /port=remote=20

>Oswald

okay, but given that I had to abort it after 20 minutes, I'm concerned about its impact on the system, and I don't know how long it's going to run, therefore my hesitation on running it on the prod system or targeted at the prod system...

sorry, I like to understand things fully before I do things on/to prod...
Oswald Knoppers_1
Valued Contributor

Re: ftp large files fail, small files succeed

>okay, but given that I had to abort it after 20 minutes, I'm concerned about its impact on the system, and I don't know how long it's going to run, therefore my hesitation on running it on the prod system or targeted at the prod system...

>sorry, I like to understand things fully before I do things on/to prod...

It won't have any impact. But just start the commando om HNATST with /port=remote=20 on a separate window and then do your FTP command. You can also compare with getting the small file.

Oswald
Hoff
Honored Contributor

Re: ftp large files fail, small files succeed

>file either.......unless I'm missing something?
>longer answer is that ftp under "server options" on both systems is started and enabled.

Without a view into the network (and diagnostics and log files and...), I'm shot-gunning. Like the rest of this thread has been doing, for a week or so now.

>as you can see below, the log file is configured, I don't know why it's not coming out...

That's close to the nub, isn't it One obvious potential answer being "you're not getting to the host you think you're connecting to".

Check the login directory for whatever user is involved here for ftp logs, too.

It's distinctly possible that there's something in the remote login that's blowing up ftp, too. Wouldn't be the first time somebody had a buggy LOGIN or SYLOGIN.

Could also be a ;32767 file version is blocking creation.

Trace from source to destination, confirm the IP addresses, confirm open ports, and why FTP is going into the weeds, and see if sftp works.

It's arguably more remarkable when ftp works than when it fails; in the recent era of network design, the default state of ftp is "not working".
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

okay, I will try that. wasn't clear that I needed to do an ftp at the same time, sorry!

anyway, an update - once the firewall issue got fixed, I was able to transfer my large file directly to hnatst from my laptop (via ftp put).

so now I can go laptop to hnaa, and laptop to hnatst, but not hnaa to hnatst (via get on hnatst nor put on hnaa).
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

Hoff - yeah, I understand the "shot in the dark" aspect of helping via forum...imagine my frustration!

I think most of what you're suggesting can be eliminated by the examples I've shown, I am connecting to the correct system, there is no disk space or version number problem, no login.com problem, etc...

haven't tried sftp yet, but could that really work where ftp doesn't (in this case)? I'll take a look, probably more out of frustration than anything else...

Robert Gezelter
Honored Contributor

Re: ftp large files fail, small files succeed

Ron,

In addition to what Hoff has mentioned, do check the Disk Quotas on the relevant disks (e.g., log file creation and other file creation can become interesting if the disk quotas are exceeded).

The command is SHOW QUOTA/USER=[identifier]

- Bob Gezelter
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

okay, here's (part of) the results of that tcptrace while the ftp was being attempted...in the interest (somewhat) of brevity, I did not include all of the output - there was a lot!

so, does this tell you anything? or, did I not include the right parts of the output, maybe?


--------------------------------------------------------------------------------

TCPtrace full display XMT packet 891 at 20-JUL-2010 10:42:32.40

IP Version = 4, IHL = 5, TOS = 00, Total Length = 192 = ^x00C0
IP Identifier = ^x0287, Flags (0=0,DF=1,MF=0),
Fragment Offset = 0 = ^x0000, Calculated Offset = 0 = ^x0000
IP TTL = 60 = ^x3C, Protocol = 6 = ^x06, Header Checksum = ^x0533
IP Source Address = 10.252.16.60
IP Destination Address = 10.252.16.75

TCP Source Port = 20, TCP Destination Port = 49207
TCP Sequence Number = 2041910197 = ^x79B513B5
TCP Acknowledge Number = 3193669793 = ^xBE5B88A1
Flags (URG=0,ACK=1,PSH=1,RST=0,SYN=0,FIN=0),
Window = 61440 = ^xF000
TCP Checksum = ^x6394, Urgent Pointer = 0 = ^x0000

3C10FC0A 3305063C 00408702 C0000045 0000 E.....@.<..3...<
A1885BBE B513B579 37C01400 | 4B10FC0A 0010 ...K...7y....[..
0D313B45 58455049 | 00009463 00F01850 0020 P...c...IPEXE;1.
20202020 20202020 20202020 2020200A 0030 .
3230312F 32323532 30312020 20202020 0040 102522/102
322D4345 442D3731 20202020 20383235 0050 528 17-DEC-2
535B2020 35333A39 323A3831 20393030 0060 009 18:29:35 [S
20202020 20202020 20205D4D 45545359 0070 YSTEM]
522C4457 522C4445 57522820 20202020 0080 (RWED,RWD,R
20666F20 6C61746F 540A0D0A 0D29522C 0090 ,R)....Total of
37393239 3138202C 73656C69 66203231 00A0 12 files, 819297
0A0D736B 636F6C62 20363933 3931382F 00B0 /819396 blocks..




--------------------------------------------------------------------------------

TCPtrace full display XMT packet 893 at 20-JUL-2010 10:42:32.40

IP Version = 4, IHL = 5, TOS = 00, Total Length = 40 = ^x0028
IP Identifier = ^x0289, Flags (0=0,DF=0,MF=0),
Fragment Offset = 0 = ^x0000, Calculated Offset = 0 = ^x0000
IP TTL = 60 = ^x3C, Protocol = 6 = ^x06, Header Checksum = ^x45C9
IP Source Address = 10.252.16.60
IP Destination Address = 10.252.16.75

TCP Source Port = 20, TCP Destination Port = 49207
TCP Sequence Number = 2041910349 = ^x79B5144D
TCP Acknowledge Number = 3193669793 = ^xBE5B88A1
Flags (URG=0,ACK=1,PSH=0,RST=0,SYN=0,FIN=1),
Window = 61440 = ^xF000
TCP Checksum = ^xF408, Urgent Pointer = 0 = ^x0000

3C10FC0A C945063C 00008902 28000045 0000 E..(....<.E....<
A1885BBE 4D14B579 37C01400 | 4B10FC0A 0010 ...K...7y..M.[..
| 000008F4 00F01150 0020 P.......




--------------------------------------------------------------------------------

TCPtrace full display RCV packet 894 at 20-JUL-2010 10:42:32.40

IP Version = 4, IHL = 5, TOS = 00, Total Length = 40 = ^x0028
IP Identifier = ^x895B, Flags (0=0,DF=0,MF=0),
Fragment Offset = 0 = ^x0000, Calculated Offset = 0 = ^x0000
IP TTL = 60 = ^x3C, Protocol = 6 = ^x06, Header Checksum = ^xBEF6
IP Source Address = 10.252.16.75
IP Destination Address = 10.252.16.60

TCP Source Port = 49207, TCP Destination Port = 20
TCP Sequence Number = 3193669793 = ^xBE5B88A1
TCP Acknowledge Number = 2041910350 = ^x79B5144E
Flags (URG=0,ACK=1,PSH=0,RST=0,SYN=0,FIN=0),
Window = 60040 = ^xEA88
TCP Checksum = ^xF980, Urgent Pointer = 0 = ^x0000

4B10FC0A F6BE063C 00005B89 28000045 0000 E..(.[..<......K
4E14B579 A1885BBE 140037C0 | 3C10FC0A 0010 ...<.7...[..y..N
0000 00000000 | 000080F9 88EA1050 0020 P.............




--------------------------------------------------------------------------------

TCPtrace full display RCV packet 895 at 20-JUL-2010 10:42:32.40

IP Version = 4, IHL = 5, TOS = 00, Total Length = 40 = ^x0028
IP Identifier = ^x895C, Flags (0=0,DF=0,MF=0),
Fragment Offset = 0 = ^x0000, Calculated Offset = 0 = ^x0000
IP TTL = 60 = ^x3C, Protocol = 6 = ^x06, Header Checksum = ^xBEF5
IP Source Address = 10.252.16.75
IP Destination Address = 10.252.16.60

TCP Source Port = 49207, TCP Destination Port = 20
TCP Sequence Number = 3193669793 = ^xBE5B88A1
TCP Acknowledge Number = 2041910350 = ^x79B5144E
Flags (URG=0,ACK=1,PSH=0,RST=0,SYN=0,FIN=0),
Window = 61440 = ^xF000
TCP Checksum = ^xF408, Urgent Pointer = 0 = ^x0000

4B10FC0A F5BE063C 00005C89 28000045 0000 E..(.\..<......K
4E14B579 A1885BBE 140037C0 | 3C10FC0A 0010 ...<.7...[..y..N
0000 00000000 | 000008F4 00F01050 0020 P.............




--------------------------------------------------------------------------------

TCPtrace full display RCV packet 896 at 20-JUL-2010 10:42:32.40

IP Version = 4, IHL = 5, TOS = 00, Total Length = 40 = ^x0028
IP Identifier = ^x895E, Flags (0=0,DF=0,MF=0),
Fragment Offset = 0 = ^x0000, Calculated Offset = 0 = ^x0000
IP TTL = 60 = ^x3C, Protocol = 6 = ^x06, Header Checksum = ^xBEF3
IP Source Address = 10.252.16.75
IP Destination Address = 10.252.16.60

TCP Source Port = 49207, TCP Destination Port = 20
TCP Sequence Number = 3193669793 = ^xBE5B88A1
TCP Acknowledge Number = 2041910350 = ^x79B5144E
Flags (URG=0,ACK=1,PSH=0,RST=0,SYN=0,FIN=1),
Window = 61440 = ^xF000
TCP Checksum = ^xF407, Urgent Pointer = 0 = ^x0000

4B10FC0A F3BE063C 00005E89 28000045 0000 E..(.^..<......K
4E14B579 A1885BBE 140037C0 | 3C10FC0A 0010 ...<.7...[..y..N
0000 00000000 | 000007F4 00F01150 0020 P.............


jjinva
Advisor

Re: ftp large files fail, small files succeed

"this is a new test system, that got set up for this upgrade."

How was it setup? Image Backup/Restore? If so maybe a network card conflict.

TCPIP> Show int/full

On both Systems and compare. I have seen this behavior with Ethernet Address conflicts from the Image restore.
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

well, guess what?? it's fixed!

they "looked again" at the ports...he apparently got confused because I had 2 nics on the same node, both configured to the same subnet (for lan failover).

when he figured out there was another port involved, he said that port was set to auto! and interestingly, it appeared to be negotiating to half...but lancp said the nic was 100 full. I didn't have console access remotely, so I couldn't check the console setting for ewa0, but could this be that "failed auto" problem that I was alluding to earlier?

thanks for everyone's input...bottom line was bad information from their network folks... (I had already suspected the half duplex problem, but.......)

SIGH!

(I'm still curious about an answer to the above question...)
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

to make it clear, the port that was set to auto was connected to hnatst, and when he set it to full (fixed), the system was disconnected for a minute (so don't do this on a prod system!) and then upon reconnection, everything worked as expected...
Hoff
Honored Contributor

Re: ftp large files fail, small files succeed

Never trust managed switches, nor the folks that run them. This not intending to impugn the vendors nor the staff, but rather the complexity of many of the managed network devices and the configurations and the UIs and the random port weirdnesses; constructs in aggregate that conspire to flummox.
Jon Pinkley
Honored Contributor

Re: ftp large files fail, small files succeed

Ron,

This is a good read:

http://www.openvms.compaq.com/openvms/journal/v12/lan_troubleshooting.html

Here is an extract that appears to be what was happening in your case:
-------------------------------------------------------------------
The following situations can lead to the wrong duplex mode being selected, as well as other problems:

A NIC is set to 100Base-TX full duplex and is attached to a switch that is auto-negotiating

In this case, the device driver for the NIC has disabled auto-negotiation because it was told to do so when the user selected 100Base-TX full-duplex mode. The switch will attempt to negotiate but will fail since the NIC isn't participating. The switch will use parallel detection. The switch will most likely select the correct speed but will select half-duplex mode even though the NIC is running full-duplex mode.
-------------------------------------------------------------------

Now that you know what adapters were involved, you should be able to see the effects of a duplex mismatch on the counters in the output of

$ mcr lancp show device /internal_counters

I would expect that your would see CRC errors being reported. Also, on the managed switch I would expect that if you could look at the counters on the port that was auto configured to half duplex, you would see late collisions, and CRC errors. When in half duplex, the switch is not expecting to see traffic arriving when it is sending data, but the full duplex side thinks that receiving and sending are decoupled, so it assumes that it can send data at any time, regardless of whether it is currently receiving data.

Jon
it depends
Ron Kaledas
Advisor

Re: ftp large files fail, small files succeed

Jon - yep, that sounds exactly like what happened. that seems to answer my question above.

good point about the lancp/internal - I had looked at sho dev/count, but forgot about /internal - wish these were not separate options! /internal tells you what you need to know...says it right there - several errors of "possible duplex mismatch"!

of course, it doesn't help when the network folks claim that "it's set to full"!...
Jon Pinkley
Honored Contributor

Re: ftp large files fail, small files succeed

Ron,

I agree that lancp show device/counters is not nearly as useful for troubleshooting as /internal. But /counters is more generic output. Internal is device driver specific, so the items you see for a DE500 is different than what you see for a DE600 even though both are 10/100 adapters. Likewise, what you see for a DEGPA-TA is different than what you see for a DEGXA-TB, even though both are 10/100/1000 adapters. In other words, the device driver writer determines what you will be able to see.

For DE602-AA and DEGXA-TA devices, the console settings are displayed, but for DEGPA-TA (the older gigabit adapter), it is not. I have included output for each of these types in the attachment.

Although it must be possible to get the console setting without access to the console, (the device driver does), it doesn't appear to be available from sys$getenv, which is limited to items defined in $STENVDEF. You can get a list with the following:

$ pipe lib/extr=$stenvdef sys$library:starlet.mlb/out=sys$output | search/nowin sys$pipe STENV$K

In other words, the following does not work:

$ mcr lancp show device /inter ewb0
... stuff removed
--- Internal Driver Counters ---
"DEGXA-TB" Device name
... stuff removed
9-APR-2010 23:29:09.41 Auto-negotiation mode set by console (EGB0_MODE)
$
$ say f$getenv("EGB0_MODE")
%DCL-W-IVKEYW, unrecognized keyword - check validity and spelling
\EGB0_MODE\
$

Jon
it depends
Jon Pinkley
Honored Contributor

Re: ftp large files fail, small files succeed

Sorry, here's the attachment.
it depends