topic Re: Ftp problem - Using split??? in Operating System - HP-UX

Ftp problem - Using split???

Adam Noble — Thu, 10 Jan 2008 17:35:21 GMT

Hi all,

We are trying to download a lot of data across the internet from another server. The files are generally around 90GB in size and the sessions are regularly timing out midway through the transfer. We are simply using ftp for this. Is using split an option to make the files more manageble. Also when you have used split on a file how do you then re-create it at the other end. I expected the man page to tell me this.

Cheers

Re: Ftp problem - Using split???

Matti_Kurkela — Thu, 10 Jan 2008 18:56:35 GMT

For example, to join a file that's been split into three parts:

cat part1 part2 part3 > whole

Or one part at a time:

mv part1 whole
cat part2 >>whole
rm part2
cat part3 >>whole
rm part3

Maybe the manual writer thought it was too simple to be worth mentioning...

MK

Re: Ftp problem - Using split???

Hasan Atasoy — Thu, 10 Jan 2008 20:23:24 GMT

hi cheers ;

on the souce machine split the file by split command .eg

split -b 1024000k filename

above command creates 1gb files

ftp them ;

on the target

cat x* > filename

Hasan

Re: Ftp problem - Using split???

VK2COT — Fri, 11 Jan 2008 00:49:39 GMT

Hello,

You already got an advice about splitting the large file.

Some additional options to consider:

a) Why not compress the files before
sending via FTP (gzip, or even better bzip2).
The idea is to decrease the total
amount of data sent over the network.

b) Replace FTP with RSYNC (with, or without
SSH support). It is a more efficient
protocol for file synchronization between
sites.

Cheers,

VK2COT

Re: Ftp problem - Using split???

Adam Noble — Fri, 11 Jan 2008 10:48:44 GMT

Thanks Matti for making me feel better about myself...Ha ha. Didn't realise it was quite as simple as that

Re: Ftp problem - Using split???

Matti_Kurkela — Fri, 11 Jan 2008 15:00:45 GMT

Sorry, my comment was not intended to slight you, but I see it's possible to read it that way too...

What I meant is, it's probably obvious to HP's Unix shell wizards who wrote the documentation of HP-UX commands.

MK