Verifying FTP'ed file whether transferred in binary or ascii mode

[Shivkumar]

Hello,

If a text file is created on windows and transferred via ftp using binary mode then we see ^M characters at
the end of each lines when opened in Unix.

I noticed if we transfer text file using ftp via ASCII mode then this ^M characters are not visible when opened
in unix vi editor.

Windows and unix text file has different end of line characters.
So why not ^M characters are observed when text files are ftp'ed from Windows to Unix
in ASCII mode ?

Second question:
If file is binary then how to verify whether it was transferred via ftp using correct binary mode ?

Thanks,
Shiv

[James R. Ferguson]

Hi Shiv:

FTP transfers that invoke ASCII mode are designed to add/subtract the carriage-return characters used on platforms like Windows as files are moved from UNIX to Windows or vice-versa. THe behavior you see is expected, desired and correct.

If you transfer in BINARY mode, no attempt is made to adjust the end-of-line character(s).

Regards!

...JRF...

[Dennis Handly]

>JRF: If you transfer in BINARY mode, no attempt is made to adjust the end-of-line character(s).

More accurately, fiddling with a binary file will corrupt a data file.

[James R. Ferguson]

Hi (again):

> Dennis: More accurately, fiddling with a binary file will corrupt a data file.

If one is only transferring a file via FTP in Binary mode, no "fiddling" should occur. If one attempts to 'vi', use 'sed', 'awk' or Perl to munge a binary file then (perhaps except for Perl's abilities) you are indeed likely to mangle what you munge.

Regards!

...JRF...

[Dennis Handly]

>JRF: If one is only transferring a file via FTP in Binary mode, no "fiddling" should occur.

That's what I meant. The binary command doesn't fiddle with the data, ascii does.

[Bill Hassell · Accepted solution]

> Windows and unix text file has different end of line characters.

ftp is a very powerful file transfer tool and is portable to many different systems. Windows and Unix are trivial examples. An ASCII file on a mainframe can be a dozen different formats including fixed line widths with no trigger character for the end of line at all. ftp's ASCII mode actually neutralizes (removes the file's internal codes) each record on both ends. Then the record is transmitted and turned into the appropriate format at the destination.

Windows mandates that a plain ASCII file has CR+LF at the end of each record. Unix requires just LF. RTE (the HP 1000) requires a 16bit number containing the length, followed by the text and then followed by the length again. Mainframes and other business machines also have specialized codes.

Although you saw ^M in vi or cat -v, what you did not see is the black spot in Windows when you send an ASCII file from Unix to Windows using binary. Notepad will show this black spot because the 2-character CR+LF is not there. If you open the file in Word or Wordpad, it looks normal but the LF by itself is defined a soft carriage return. This has a different meaning than a hard carriage return (CR+LF) for word processors.

So ASCII is a very specific translator and should only be used with true ASCII files. BINARY turns off all translation and sends exactly what in the source file. Whether this is useful on the destination system depends on what is in the file and the process used to read it.

[Steven E. Protter]

Shalom Shiv,

A secondary script can use the file command against it. The list can be gathered with a find +mtime command.

Or use sftp and stop caring. Secure shell always gets it right.

SEP