topic Re: ftp problem with larges files. in Operating System - HP-UX

ftp problem with larges files.

mornet — Tue, 15 May 2001 09:00:38 GMT

Hello,
Excuses me for the english.
I have a probleme with the transfert of larges files (70 Mo) using ftp, from a machine(remote host A) to a server 9000 under HP-UX.11.00.(local host B).
the transfert from A to B begins at 17h46 and stop at 20h58. the syslog.log file of the local host says "exiting on signal 14". and on A there is a message:"connection reset by peer, connection closed by remote host"(B)
is it a question of timeout?. On B, the idle parameter= 900.(max =7200 sec)
What the probleme? should I cut the larges files before the ftp? or there is a solution to transfert the larges files?

Thanks for all
Thomas

Re: ftp problem with larges files.

Ravi_8 — Tue, 15 May 2001 09:12:16 GMT

hi,

compress the file at A and use ftp

Re: ftp problem with larges files.

Vincenzo Restuccia — Tue, 15 May 2001 09:18:45 GMT

#gzip file
or
#compress file
I you have multiples files tar cvf and after compress or gzip.

Re: ftp problem with larges files.

mornet — Tue, 15 May 2001 10:13:07 GMT

I forgot to say that the file is already compressed (.tar)
so there is a pb on the HP server? or on the remote host A?
Thanks for all

Thomas

Re: ftp problem with larges files.

Juan González — Tue, 15 May 2001 12:59:17 GMT

Hi Mornet,
I suposse that the problem is in the remote host A because the host B logs that connection closed by peer.
Host A might have a parameter like session max time or something.
If host A is a Unix system you can use split to break the file in smaller files.

Best regards
JGM

Re: ftp problem with larges files.

Vincenzo Restuccia — Tue, 15 May 2001 13:01:36 GMT

Check you frees inodes with:
#sar -v 5 5

Re: ftp problem with larges files.

Stefan Schulz — Tue, 15 May 2001 13:20:55 GMT

Just a small sidenote:

Tar uses no compression, so a *.tar file is not compressed. You have to use gzip or compress to reduce this file in size. Which should give you a *.tar.gz or *.tar.Z file.

Hope this helps. Stefan

Re: ftp problem with larges files.

John Bolene — Tue, 15 May 2001 13:25:02 GMT

70 Mo? Maybe you meant 70 Meg which is not a big file. Now if you meant 70 Gig, that will take a while and is a large file. We regularly send files in the 200 to 500 Meg range and have no problems. If you have trusted rlogin access to the remote system, you can use rcp.

It may be that you are running out of space in the file system that you are sending the file to. This space can either be inodes or file space. You can do a bdf -i to see if space is available.

Re: ftp problem with larges files.

Vinoth — Tue, 15 May 2001 13:27:11 GMT

Hi,

Try rcp command instead of ftp.
Vinoth

Re: ftp problem with larges files.

mornet — Tue, 15 May 2001 13:37:37 GMT

first, ok for the "compressed" .tar file, it was a error from me. second, i know that 70 Mo is not a "large file" but it's not a small file.
for information, the transfert is made by a ISDN line, and after a another test, the ftp transfert seems to be good for the moment. It was may be a ISDN line probleme between the 2 ISDN routers.
Thanks for all and for your speed to answer.

Re: ftp problem with larges files.

Valkiria Santos — Tue, 15 May 2001 19:29:38 GMT

Hi,

Look at in both machines if you have the arpa lan and ftp patches news installed.

God luck,
Valkiria

Re: ftp problem with larges files.

Abel Berger — Tue, 15 May 2001 19:44:28 GMT

If you have a tape drive in each server ( A and B ), makes a copy for tape dat and later it lowers in the B server.

Commands :

tar -cvf ( UP A server ).
tar -xvf ( down in B server ).

Re: ftp problem with larges files.

Shailesh Bhamre — Wed, 16 May 2001 12:05:40 GMT

Try using command compress & then send it thru FTP

Re: ftp problem with larges files.

echong — Wed, 16 May 2001 13:08:13 GMT

Try compress it with different bit or different compression software. I have experience that the compression procedure insert certain binary sequence into your compressed file that actually stop the ftp process. So the ftp always timeout after transfering exact number of bytes. It also affect OmniBack backup. Try compress it with different bits number or even try ftp on the uncompressed file.

Re: ftp problem with larges files.

John Hall — Wed, 16 May 2001 14:29:23 GMT

Here's an approach noone thought of:

1) Compress the file. In this example I will call
the compressed file "myfile.tar.Z".

2) Use split to break the file into smaller files:

split -b 5m myfile.tar.Z mysplit

For a 70MB file this will create 14 files,
each one 5MB in size. They will be called
mysplitaa, mysplitab, mysplitac, etc.

3) Ftp these small files (in binary) from A to B.
If your problem is a timing issue, in that 70MB
takes too long to transfer, these 5MB files should
ftp Ok.

4) Once all files have been ftp'ed from A to B
rejoin them:

cat mysplit* > myfile.tar.Z

5) Now you can uncompress myfile.tar.Z and untar
it on B. (Don't forget to delete the small 5MB files
after you have rejoined them into myfile.tar.Z)

I hope this helps!

Re: ftp problem with larges files.

rick jones — Wed, 16 May 2001 16:06:14 GMT

It is quite possible for an ISDN link to become congested. Perhaps even congested enough to cause a TCP connection to hit its maximum retransmission limit and abort.

If the "problem" starts happening again, I would suggest you look at the TCP statistics on the sending system - in particular look and see if/how the TCP retransmit stats are increasing and what percentage of the number of data segments they happen to be.

On the receiving side, you would look for "out of order" segments and perhaps duplicate segments. You might also look for TCP checksum failures.

It might also be nice to know what the ping times are between the two systems while the FTP transfer is taking place. Try a ping with a count of 100 and see what the loss rate happens to be.

Re: ftp problem with larges files.

Abel Berger — Thu, 17 May 2001 18:17:48 GMT

Hi Mornet

For me you can be having problems in the network.

It verifies first if the lan adapter and switch are talking in fullduplex mode and autonegociation off.

Later it tries to change of switch port.

Later test using a new network cable.

And finally the problem can be the network lan adapter, change it.

Send to me the folowing commands :

# lanadmin / lan / display
# lanadmin -x
# lanadmin -I
and
# netstat -I

You must verify the fields "inboud errors", "outbound errors" and "FCS
errors ( frame check sequence )"

I wait to have helped !