Need FTP scripts

[vthiru]

Hello Admins,

I would like to ftp about 40GB of data from my server to ftp server. There are lots and lots of folders, sub folders and files present. Tried to ftp, but it is not sending directories. There is no space left in the drive to compress the parent directory and ftp it. Please help me with scripts to automate the ftp.

Any Help is greatly appreciated.

Thanks,
vthiru

[Rasheed Tamton]

Hello Vthiru,

You could use tar with gzip and it will produce a tar archive file. You can ftp/rcp that archive to the destination system and extract it from there.

Regards,
Rasheed Tamton.

[Rasheed Tamton]

>There is no space left in the drive to compress the parent directory and ftp it.

In that case, you could copy the compressed file to another mountpoint where there is enough space.

tar cvf - ./home/vthiru/test |gzip - > /tmp/test.tgz

ftp test.tgz to the destination server

gzip -dc test.tgz |tar -tvf -
gzip -dc test.tgz |tar -xvf -

[vthiru]

There is not enough space left in the drive to compress the folder. Space left is less than 1GB.

Thanks,
vthiru

[vthiru]

Thanks for the response. But, there is only one drive present. I don't have choice of compressing the folder to a different mount point. That is the reason I am looking for some FTP scripts to upload file to my ftp server.

Thanks,
vthiru

[Rasheed Tamton]

In that case, is it possible for you to use rcp/scp.
rcp -rp /dirname remotehost:/dirname/

or
if you can get hold of a ftp GUI application you would be able to just drag and drop to the remote-system/dir. as in the Windown Filemanager

Regards.

[Rasheed Tamton]

Also check wget

http://www.gnu.org/software/wget/

Rgds.

[Steven E. Protter · Accepted solution]

Shalom,

scp -rp * hostname:/directory

This will take subdirectories and put them on the target server.

It is likely that the target system has sshd daemon running, this is a default with Linux.

Also. openssh which includes sftp/ssh/scp is more secure. Authentication and data stream is encrypted. That makes it a little slower though.

SEP

[Magnus Andersen]

I too would suggest scp as the best option. If you want to use an ftp client to do it you should be able to use lftp.

HTH,
Magnus

[Jimmy Vance]

Another option is to use a combination of netcat and tar.

move data from machine a to machine b

on machine a

change to the directory you want to start from

# tar -cf - . | netcat -w 3 -l -p 5432

on machine b

change to the directory where you want the files to go

# netcat -w 5 a.example.com 5432 | tar -xvf -

The command sets up a tar process on machine a that waits for a tcp connection on port 5432. When a client (machine b) connects, tar simply sends all the data through that tcp port. No temporary files are required. On machine b, another tar process reads from the network via netcat, writing
the data to the disks as it streams over the network


Some flavors of Linux netcat is called nc

[Steven Schweda]

Is FTP the only way you can access the
destination server? With rsh (or ssh), it's
normally possible to construct a "tar"
pipeline (with or without compression), and
do the job with no need for temporary disk space.

Wget (-r) can be useful, but you'd need to
run it on the destination system. (And if
you can do that, you could probably use the
rsh + "tar" pipeline method.)

> [...] Tried to ftp, but it is not sending
> directories. [...]

A typical FTP client does what you tell it to
do. I have not seen one which can read
minds.

> Please help me with scripts to automate the
> ftp.

That's not the easy way.

[macosta]

Also, if FTP is a hard requirement, and you have ksh installed, try hardfeed:
http://www.unix.com/shell-programming-scripting/9174-recursive-ftp-here-last.html

[Mark Ellzey]

Vthiru,

Why not use NFS? You could mount a filesystem from your ftp server and use standard cp/tar/cpio utilities to move the data. You also wouldn't be subject to ftp's one-file-at-a-time limitation.

Regards,
Mark

[Steven Schweda]

> Why not use NFS? [...]

Lack of administrative access on one system
or the other?

Many things are possible. Some are easier
than others. Without more information on the
environment and its limitations, it's tough
to do more than guess at an optimal solution.

I normally assume that a questioner who, in
the face of repeated requests, fails to
supply an adequate explanation of the
problem, probably doesn't really care much
about getting a useful answer.