System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Verifying FTP'ed file whether transferred in binary or ascii mode

 
SOLVED
Go to solution
Shivkumar
Super Advisor

Verifying FTP'ed file whether transferred in binary or ascii mode

Hello,

If a text file is created on windows and transferred via ftp using binary mode then we see ^M characters at
the end of each lines when opened in Unix.

I noticed if we transfer text file using ftp via ASCII mode then this ^M characters are not visible when opened
in unix vi editor.

Windows and unix text file has different end of line characters.
So why not ^M characters are observed when text files are ftp'ed from Windows to Unix
in ASCII mode ?

Second question:
If file is binary then how to verify whether it was transferred via ftp using correct binary mode ?

Thanks,
Shiv
6 REPLIES
James R. Ferguson
Acclaimed Contributor

Re: Verifying FTP'ed file whether transferred in binary or ascii mode

Hi Shiv:

FTP transfers that invoke ASCII mode are designed to add/subtract the carriage-return characters used on platforms like Windows as files are moved from UNIX to Windows or vice-versa. THe behavior you see is expected, desired and correct.

If you transfer in BINARY mode, no attempt is made to adjust the end-of-line character(s).

Regards!

...JRF...
Dennis Handly
Acclaimed Contributor

Re: Verifying FTP'ed file whether transferred in binary or ascii mode

>JRF: If you transfer in BINARY mode, no attempt is made to adjust the end-of-line character(s).

More accurately, fiddling with a binary file will corrupt a data file.
James R. Ferguson
Acclaimed Contributor

Re: Verifying FTP'ed file whether transferred in binary or ascii mode

Hi (again):

> Dennis: More accurately, fiddling with a binary file will corrupt a data file.

If one is only transferring a file via FTP in Binary mode, no "fiddling" should occur. If one attempts to 'vi', use 'sed', 'awk' or Perl to munge a binary file then (perhaps except for Perl's abilities) you are indeed likely to mangle what you munge.

Regards!

...JRF...
Dennis Handly
Acclaimed Contributor

Re: Verifying FTP'ed file whether transferred in binary or ascii mode

>JRF: If one is only transferring a file via FTP in Binary mode, no "fiddling" should occur.

That's what I meant. The binary command doesn't fiddle with the data, ascii does.
Bill Hassell
Honored Contributor
Solution

Re: Verifying FTP'ed file whether transferred in binary or ascii mode

> Windows and unix text file has different end of line characters.

ftp is a very powerful file transfer tool and is portable to many different systems. Windows and Unix are trivial examples. An ASCII file on a mainframe can be a dozen different formats including fixed line widths with no trigger character for the end of line at all. ftp's ASCII mode actually neutralizes (removes the file's internal codes) each record on both ends. Then the record is transmitted and turned into the appropriate format at the destination.

Windows mandates that a plain ASCII file has CR+LF at the end of each record. Unix requires just LF. RTE (the HP 1000) requires a 16bit number containing the length, followed by the text and then followed by the length again. Mainframes and other business machines also have specialized codes.

Although you saw ^M in vi or cat -v, what you did not see is the black spot in Windows when you send an ASCII file from Unix to Windows using binary. Notepad will show this black spot because the 2-character CR+LF is not there. If you open the file in Word or Wordpad, it looks normal but the LF by itself is defined a soft carriage return. This has a different meaning than a hard carriage return (CR+LF) for word processors.

So ASCII is a very specific translator and should only be used with true ASCII files. BINARY turns off all translation and sends exactly what in the source file. Whether this is useful on the destination system depends on what is in the file and the process used to read it.


Bill Hassell, sysadmin
Steven E. Protter
Exalted Contributor

Re: Verifying FTP'ed file whether transferred in binary or ascii mode

Shalom Shiv,

A secondary script can use the file command against it. The list can be gathered with a find +mtime command.

Or use sftp and stop caring. Secure shell always gets it right.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com