Simple Automated FTP Scripts

[Alex St. Amand_1]

I am in need of an FTP script or utility to transfer a file back and forth from a vendors server on a daily basis. The script/utility would need the capability of checking if the file exists and if it is complete before proceeding (possibly by checking the file size or date).

I am new to Unix scripting, and I admit this is above my abilities (I am an expert from the NT world). However, I am capable of modifying any current script to work.

I am humbly hoping there is a current good script out there or a good utility to accomplish this task.

Thanks Everyone.

[Steven Sim Kok Leong]

Hi,

For an unsecure implementation, use a combination of remsh and rcp.

For a secure implementation, use a combination of ssh and scp.

You can remote shell to your server to execute the shell script. In the shell script, perform the necessary checks using [ -e filename ]. To check if the file is complete, use lsof to check if the file has stopped being written into. Subsequently perform remote copy to copy the file back to your client.

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com

[Tommy Palo]

If you want to use ftp in an automated way it was discussed in the following thread:
http://forums.itrc.hp.com/cm/QuestionAnswer/1,1150,0xffe5f841489fd4118fef0090279cd0f9,00.html

To run it daily you could use "cron".
Look at the manpages for "test" to find out how to check for if the file exists.

[Hugh Trevelyan]

The example script below is a bit crude but it does work.

To get a remote file:

ftp_copy_file.sh user get

To push a local file to a remote host:

ftp_copy_file.sh user put

#!/bin/ksh

function usage {

print "Usage: $APP_NAME <[get|put] "

}

function ftp_get_remote_file {

ftp -i -n $REMOTE_HOST <

[Shannon Petry]

The beauty of UNIX is that there so many ways of doing what you want done.

As the purpose of the forums is not to have people work for you for free!, It may be worth a consulting call to accomplish "EXACTLY" what you need!!!

I have at sites setup a mixture of scripts and cron jobs to do what you need, but these are very specific. Some encrypt data, some make tarballs of lots of data, some compress but all move data from site to site.

Some ideas were mentioned about using ssh, and scp for secure connections, and lsof to check for open files. remember that lsof, and ssh are NOT included with HP-UX so you must install them on your own!

The easiest way to do what you need, but very insecure is to use a $HOME/.netrc file, and create a user for this job. Give them access to run a cron job, and schedule as that user a cron job which is "/usr/bin/ftp $dest". the .netrc file allows you to create a macro called init which is automatically executed at each host ftp's to. Using this method, the data could be transferred.

there is a large problem with you not owning the vendors server though, and that is how will you control the test to see if the file is complete? This must be executed on the vendors server!

Some easy ref's for you would be...
man netrc
man cron
man ftp

Regards,
Shannon

[James R. Ferguson]

Hi Alex:

You posted this yesterday at least twice. I offered you a reply in this thread (named differently than this one, but containing the same query:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,1150,0xfb7adfe5920fd5118fef0090279cd0f9,00.html

...JRF...

[Shannon Petry]

Hey JRF, I was gonna say the same thing, but descided to be nice. the guy posted the same message 3 times.

Well, that's NT thinking I guess. Press Okay, Press "Yes", Press "Yes", Press "Yes", Press "Yes" and then perform the task! LOL

Shannon

[Andrew F Greenwood]

I've done quite a bit with FTP automation recently, and I would recommend using a combination of both command redirection and .netrc.

If you use .netrc with "macdef init" for the whole automation you do limit yourself. You can only have one FTP session going at a time.

If you use input redirection (using here statements (<<-EOF) or simply echoing a bunch of commands into | ftp -iv $REMHOST, then you compromise security by having unencrypted IDs and passwords.

Here is what I have been using:

Use the .netrc to automate the log on. You can have multiple remote hosts in the .netrc, and it hides the log on info.

You can then use input redirection to script the remaining commands. This allows you to have multiple FTP sessions to multiple remote hosts without fear of conflict.

The only downside is that FTP doesn't give non zero return codes if any of the commands fail. You can try using a Korn shell co-process to give you greater interaction with the FTP session. I've attached my first attempt at it if you are interested. (Ignore the stdlistfile command - that is something else I use for handling Maestro.)

Hope that helps.

[A. Clay Stephenson]

Hi Alex:

When I have to do ftp scripting nowadays, I use perl with the Net::FTP module (available for download at www.perl.org/CPAN). It really makes FTP operations very simple. Any error conditions and timeouts are easy to test for and set. The other advantage is that unlike shell script based solutions, your perl scripts can run unchanged on NT and W2K boxes if you install the freely available product ActivePerl (www.activeperl.com) .

It's usually something as simple as this (with error checking omitted):

use Net::FTP;

$ftp = Net::FTP->new("remhost.xxx.yyy", Debug => 0);
$ftp->login("anonymous",'cstephen@mama.com');
$ftp->cwd("/Downloads");
$ftp->get("Testfile.TXT");
$ftp->quit;


Regards, Clay