Networking
cancel
Showing results for 
Search instead for 
Did you mean: 

cannot transfer CSV Files from linux to windows correctly

SOLVED
Go to solution
'chris'
Super Advisor

cannot transfer CSV Files from linux to windows correctly

hi

I have a big problem and need really help.
I should transfer CSV files from linux to the WIN2000 ftp server via perl script, but all files after transfer are currupt.
and no matter how I ftp these files:
perl script, from shell Konsole,gftp,
passive transfer,ascii or binary.
they are always corrupt after ftp transfer.

should I encode or convert to the MS Windows format and how it works using perl ?

I can only transfer via ftp these files.

windows to windows transfer all files correctly.

kind regards
chris
27 REPLIES
harry d brown jr
Honored Contributor
Solution

Re: cannot transfer CSV Files from linux to windows correctly

Because windows is stupid, you probably have to do a unix2dos conversion first before transfering them. It adds a carriage return or line feed (I forgot which is which) (^M) to the end of each line.
Live Free or Die
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

hi

I've tried to convert with unix2dos, but still doesn't work.

greetings
chris
harry d brown jr
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

on your windoze box try using "rcp":

C:\stupidmachine\ rcp -b linuxbox.root:/some directory/* .

your linux box has to "trust" the windoze box (man that is scary).

live free or die
harry
Live Free or Die
Victor BERRIDGE
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Hi,
How about considering the use of samba?
That should work as I do something of the sort for my SAS users who need to recuperate the csv file into excel from a shared drive mounted on their PCs.

All the best
Victor
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

I can use ftp only.
I cannot use samba over the internet.
Victor BERRIDGE
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Could it be something to do with charset? Like for mail?
set crt=21
set encoding=8bit
set charset=iso-8859-1

Just thoughts...

Good luck

Victor
harry d brown jr
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Are you ftp'ing from windows or TO windoze? If TO, what FTP server on windoze are you using?

Have you tried to move a single file manually to windoze, then move it back to linux to see if it is actually corrupted? (of course use BINARY mode both ways).

What errors on windoze do you get?

live free or die
harry
Live Free or Die
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

maybe yes and I think so, but how to encode these files using perl ?
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

hi harry

I cannot control the remote server.

I transfer all files from LINUX TO WINDOWS.

on the remote site it's:
220 szhstorage Microsoft FTP Service (Version 5.0).

if I do ftp transfer from any windows maschine to this remote Mickysoft server all files are OK.

but all files transfered from linux are corrupt.

I've tried to move this file from linux to windows, move back to linux, transfer again, but still corrupt.
Victor BERRIDGE
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Could you describe what corruption you have, since you transmetted file is pur ascii how does it get modified?

Courage!
Ganesh Babu
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

could u post a corrupted file and a good one???

Ganesh
Bruno Ganino
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

good file looks:
---------------------------------------------------------------------------------------
I,ZZ0387778,Name Firstname,USD,street 166,city,,,,B:266.27.07,3654,USD,P:351.15.79,,,DE,USD107,
I,ZZ0387780,Name Firstname,USD,street,city,,,,B:792.58.50,6344,USD,P:485.29.85,,,DE,USD001,
I,ZZ0274516,Name Firstname,USD,street 2,city,,,,B:834.33.23,3292,USD,P:284.89.60,,,DE,USD033,
I,ZZ0558613,Name Firstname,USD,street 102,city,,,,,8032,USD,P:383.31.96,,,DE,USD999,
---------------------------------------------------------------------------------------

bed file (after ftp transfer) looks:
---------------------------------------------------------------------------------------
I,ZZ0387778,Name Firstname,USD,street 166,city,,,,B:266.27.07,3654,USD,P:351.15.79,,,DE,USD107,
I,ZZ0387780,Name Firstname,USD,street,city,,,,B:à à dîà à k£ð9à šà à ü±Réà Zùüáà à à þ
I,ZZ0274516,Name Firstname,USD,street 2,city,,,,B:834.33.23,3292,USD,P:284.89.60,,,DE,USD033,
I,ZZ0558613,Name Firstname,USD,street 102,city,,,,,8032,USD,P:383.31.96,,,DE,USD999,
---------------------------------------------------------------------------------------

with xftp works, but I cannot use it in my case.
what can xftp, what the others cannot do ?
how to to simulate xftp using perl ?
may
Ganesh Babu
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Ganesh Babu
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

i am not sure whether this will work.. but try transfer of file in binary mode itself..

Because in the More information section of my previous link says

The FTP protocol offers two types of transfers: binary and ASCII. In a binary download (which is always used by the ITC), the FTP server sends to the client the exact contents of a file. In a binary upload, the FTP server stores exactly what the client sends. This is desirable behavior for most types of files, such as executable programs, images, word processor documents, and so on.

Ganesh

'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

hi

I've tried already to transfer binary, but it doesn't work either.

I get these files from a IBM machine and they are encoded in EBCDIC, I think.
I should rename files and send automatically via ftp to the remote windows machine.

the solution can be, to encode using perl from EBCDIC to the windows standard ?
Bruno Ganino
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Hi Chris, read this document. It is interesting...
http://www.microsoft.com/technet/itsolutions/migration/linux/mvc/lintowin.mspx

HTH
Bruno
Torino (Turin) +2H
Victor BERRIDGE
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

Hi chris,
Is your IBM an mainframe mvs/OS390 ?
I would start by looking at the files transmetted from IBM to linux.
Is the file what you expect? when you look at it with vi , all seems mormal?

There are 2 cases with tranfer to mainframes:
ascii/ebcdic and block format
You can ask the mainframe to translate to ascii
From /to a GCOS system I had to use dd:
dd if= of= cbs= conv=unblock
I would try with the option conv=unblock, then pass this file through ux2dos> "file_for_win"
If I can think of something else I will let you know, keep us informed

Good luck and all the best
Kind regards
Victor
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

hi

OS from the IBM maschine is AS400.
If I look this file with vi or kwrite looks always fine.

and what about lenght for fixed block size ?

The files, which I get from AS400 have each time differnt size.

I played a little bit with language settings from linux and for a short time all transfers were correctly, but after reboot like before.

and xftp from
http://www.llnl.gov/ia/xftp.html
transfer all files always correctly.

knows someone, if I can use xftp from shell Konsole or use it in combination with a perl script ?

greetings
chris
Jon Finley
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

You are going to have to convert from EBCDIC to ASCII.

MS
http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q216/3/99.ASP&NoWebContent=1

Linux
Look at the man page for dd On Solaris, you can use the conv switch to have dd translate ebcdic to ascii.
http://www.uwsg.iu.edu/usail/man/solaris/dd.1.html

Jon
"Do or do not. There is no try!" - Yoda
Victor BERRIDGE
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly


Since to file is ok on your linux I believe ther is no problem from IBM side (have you tried conv=unblock?).
I noticed some time ascii files get corrupted when big amounts in MS...
Try to download a winzip(compatible like zip) on your linux and compress and transfer in binary

What does that give you?

All the best
Victor
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

conv=unblock doesn't help.

transfer of zip files doesn't help in my case.
I must transfer ascii these files, otherwise cannot be processed on the remote site.
Jon Finley
Honored Contributor

Re: cannot transfer CSV Files from linux to windows correctly

hmmm... on the linux side, in FTP, try setting type to EBCDIC.

Jon
"Do or do not. There is no try!" - Yoda
'chris'
Super Advisor

Re: cannot transfer CSV Files from linux to windows correctly

and how can I convert the file to EBCDIC ?