topic Re: Need FTP scripts in Operating System - Linux

Need FTP scripts

vthiru — Mon, 19 May 2008 06:03:59 GMT

Hello Admins,

I would like to ftp about 40GB of data from my server to ftp server. There are lots and lots of folders, sub folders and files present. Tried to ftp, but it is not sending directories. There is no space left in the drive to compress the parent directory and ftp it. Please help me with scripts to automate the ftp.

Any Help is greatly appreciated.

Thanks,
vthiru

Re: Need FTP scripts

Rasheed Tamton — Mon, 19 May 2008 06:58:36 GMT

Hello Vthiru,

You could use tar with gzip and it will produce a tar archive file. You can ftp/rcp that archive to the destination system and extract it from there.

Regards,
Rasheed Tamton.

Re: Need FTP scripts

Rasheed Tamton — Mon, 19 May 2008 07:07:08 GMT

>There is no space left in the drive to compress the parent directory and ftp it.

In that case, you could copy the compressed file to another mountpoint where there is enough space.

tar cvf - ./home/vthiru/test |gzip - > /tmp/test.tgz

ftp test.tgz to the destination server

gzip -dc test.tgz |tar -tvf -
gzip -dc test.tgz |tar -xvf -

Re: Need FTP scripts

vthiru — Mon, 19 May 2008 07:10:28 GMT

There is not enough space left in the drive to compress the folder. Space left is less than 1GB.

Thanks,
vthiru

Re: Need FTP scripts

vthiru — Mon, 19 May 2008 07:12:33 GMT

Thanks for the response. But, there is only one drive present. I don't have choice of compressing the folder to a different mount point. That is the reason I am looking for some FTP scripts to upload file to my ftp server.

Thanks,
vthiru

Re: Need FTP scripts

Rasheed Tamton — Mon, 19 May 2008 07:26:08 GMT

In that case, is it possible for you to use rcp/scp.
rcp -rp /dirname remotehost:/dirname/

or
if you can get hold of a ftp GUI application you would be able to just drag and drop to the remote-system/dir. as in the Windown Filemanager

Regards.

Re: Need FTP scripts

Rasheed Tamton — Mon, 19 May 2008 07:59:05 GMT

Also check wget

http://www.gnu.org/software/wget/

Rgds.

Re: Need FTP scripts

Steven E. Protter — Mon, 19 May 2008 08:57:26 GMT

Shalom,

scp -rp * hostname:/directory

This will take subdirectories and put them on the target server.

It is likely that the target system has sshd daemon running, this is a default with Linux.

Also. openssh which includes sftp/ssh/scp is more secure. Authentication and data stream is encrypted. That makes it a little slower though.

SEP

Re: Need FTP scripts

Magnus Andersen — Mon, 19 May 2008 11:31:21 GMT

I too would suggest scp as the best option. If you want to use an ftp client to do it you should be able to use lftp.

HTH,
Magnus

Re: Need FTP scripts

Jimmy Vance — Mon, 19 May 2008 13:06:18 GMT

Another option is to use a combination of netcat and tar.

move data from machine a to machine b

on machine a

change to the directory you want to start from

# tar -cf - . | netcat -w 3 -l -p 5432

on machine b

change to the directory where you want the files to go

# netcat -w 5 a.example.com 5432 | tar -xvf -

The command sets up a tar process on machine a that waits for a tcp connection on port 5432. When a client (machine b) connects, tar simply sends all the data through that tcp port. No temporary files are required. On machine b, another tar process reads from the network via netcat, writing
the data to the disks as it streams over the network

Some flavors of Linux netcat is called nc

Re: Need FTP scripts

Steven Schweda — Mon, 19 May 2008 13:22:03 GMT

Is FTP the only way you can access the
destination server? With rsh (or ssh), it's
normally possible to construct a "tar"
pipeline (with or without compression), and
do the job with no need for temporary disk space.

Wget (-r) can be useful, but you'd need to
run it on the destination system. (And if
you can do that, you could probably use the
rsh + "tar" pipeline method.)

> [...] Tried to ftp, but it is not sending
> directories. [...]

A typical FTP client does what you tell it to
do. I have not seen one which can read
minds.

> Please help me with scripts to automate the
> ftp.

That's not the easy way.

Re: Need FTP scripts

macosta — Tue, 20 May 2008 06:08:19 GMT

Also, if FTP is a hard requirement, and you have ksh installed, try hardfeed:
http://www.unix.com/shell-programming-scripting/9174-recursive-ftp-here-last.html

Re: Need FTP scripts

Mark Ellzey — Wed, 21 May 2008 17:53:31 GMT

Vthiru,

Why not use NFS? You could mount a filesystem from your ftp server and use standard cp/tar/cpio utilities to move the data. You also wouldn't be subject to ftp's one-file-at-a-time limitation.

Regards,
Mark

Re: Need FTP scripts

Steven Schweda — Wed, 21 May 2008 21:06:47 GMT

> Why not use NFS? [...]

Lack of administrative access on one system
or the other?

Many things are possible. Some are easier
than others. Without more information on the
environment and its limitations, it's tough
to do more than guess at an optimal solution.

I normally assume that a questioner who, in
the face of repeated requests, fails to
supply an adequate explanation of the
problem, probably doesn't really care much
about getting a useful answer.