how to activate a script on a server after receiving a file thru ftp

[Laurent Laperrousaz]

Hi!

I would like to activate a script or a command (event-driven) when a file has been received on my server.
I don't want to poll for the file, I would like ftp server to trigger the script when finishes receiving the file.
Is that possible ?

Thanks

Laurent

[Dave La Mar]

Laurent -
There are several ways to do this dependent upon the platform the ftp is going to.
We commonly will ftp a file from our mainframe to HP-UX. Following a successful completion of the file transfer, we'll kick off another ftp job to perform a "site exec" on HP-UX to invoke a script.
To do the same to an NT platform, the NT platform need to allow remote exec, which is a low cost add on for NT.
Another option we use on HP-UX is to immediately execute a script while the file transfer is tranpiring. That script check the byte count on the file being transferred; when it no longer changes the script executes. (not preferred over the first one mentioned.)

Best of luck.
And please give us a little more info on the platforms you are dealing with on this question.

Regards,
dl

[harry d brown jr]

Not unless you want to modify FTP.

Polling for the existence of files doesn't consume a lot of cycles, unless of course you don't have the script/program sleep for a period of time before starting anew.

An important caution is that the sender should send you a file, then a file with the same name but with a different SUFFIX, once the original file has been correctly and entirely sent. This prevents any script/program from copying/processing a file BEFORE it has completed the file transfer. If you don't adhere to this, then you will need to use something like lsof to determine if anyone has it open before processing it.


live free or die
harry

[John Meissner]

If you are transfering from a unix server to a unix server - and you are using a script i'm assuming to ftp the file....
then you could install "expect" on your 1st server (ftp from). Install expect and then have use it to create a script that your ftp scrpit, when complete, will call. Expect will telnet (or ssh) to your other server and can launch your script.

expect can be found at:
http://expect.nist.gov/

I'm pretty experienced with using it so if you have any questions or want an example script I'd be more than happy to lend a hand.

[Dave La Mar]

Laurent -
Assuming you are going unix to unix here is one of the many ways to do this -

#! /bin/ksh
ftp -n -v << endl >> /home/dlamar/work/ftp.log # Start ftp and create a log file for verification of completion.
open remotename # Open the destination.
user username userpassword # Log in with user name and passwd (case sensitive).
ascii # Transfer mode needs to be ascii
put /home/dlamar/work/rdrpause # Put your file.
quit # Quit after the put completes
endl # End the logging

echo $? | read RC
if [ $RC -gt 0 ]
then
tail -14 /home/dlamar/work/ftp.log | mailx -s "FTP Failure results" dlamar@gottschalks.com # Mail results to a user
else
remesh remotename -l root -n remote_script_name >> remsh_results_file
fi

Of course, you can error check on the remesh as well. Do not place the comments in the actual script.

Again, best of luck.

dl

[Jordan Bean]

If this trigger is specific to a file from a client you control, then use the SITE EXEC feature.

Otherwise, if you want to process every file put to the server, then it may be best to have a daemon script watch the xferlog for files put into specific locations. (ftpd -i enables inbound xferlog.)

If the transfers are sparse and you don't want another daemon, then it possible to use a inetd wrapper to run ftpd and the script concurretly. In this senario it may be best to use the command log (-L, syslog LOCAL5) rather than the xferlog so you know when to stop watching. The attached script illustrates the concept.

[Jos Francois]

We had the problem of files coming in from different outside locations (i.e. authorities, customers,). Some of those files are large files and picking the file up before end of transmission was a real danger. We could not force our outside partners to put the file under a nickname during transmission and rename the file afterwards. Launch e rexec or remsh from an outside system is against our security policy.

Next statement uses fuser: the statement contains the process id from ftpd as long as the file is open during transmission. We poll a few directories every minute and check if the file is still open. If so, do nothing and wait for the next poll.

for FILE in $FILELIST #... File by file
do
if [ "$(/usr/sbin/fuser $FILE 2>/dev/null | cut -f2 -d ":")" = "" ] #... Check on open file
then

[Jordan Bean]

Jos brings up a good point. I checked again to find that ftpd logs the upload as it happens, not after. So my script must be revised to wait until the files are closed... or keep a list of files and wait until the ftp session finishes before processing each file.

[David Ritchie]

I have ran into this problem in the past - unfortunately there is only two ways that I can think of to accomplish this:

1) Use a tool like lsof to see if the file is still open - if it is, sleep and
recheck.

-or-

2) place what we call a '.rdy'
(i.e. ready) file after the file we are wanting to pull is tranferred. Scan for the
existance of the ready file,
not the file that is transferred. Delete the ready
file when you have finished pulling the data file down.

In my experience, option 2 is
easier.

[Jordan Bean · Accepted solution]

Revised ftpd wrapper script attached.

It should be sufficient to wait for ftpd to terminate before processing each file uploaded during that session.

You must do three things before using this script:

Make sure you have an entry for local5 in syslog.

Make sure ftpd is run with -L.

Scrutinize the script! It hasn't been thoroughly tested. It's up to you to make it more selective about the files it will process.

Disclaimer:

Security has not been considered, so use with caution.

[Laurent Laperrousaz]

thanks to Jordan for the wrapper!
I think that's the best way to achieve my goal.

As I said in my initial question, I didn't want to poll for the files to arrive because in my case, the files received are in fact transactions which are to be processed quickly. Then the polling need to be short -> cpu consumer!

Thanks to all!

I gave points but they don't appear in the forum!

BUG in the forum??

[John Meissner]

Laurent.... yeah.. it's a bug... you have to assign them twice before they "stick"

[Laurent Laperrousaz]

I submitted at least 5 times but nothing changed!

sorry for everybody

my browser is OPERA 7.11

I wanted to use Netscape but I have 2 accounts (one is Laurent Laperrousaz and the other is Laperrousaz) and Netscape comes up with the other account set so I can't assign points!
Then I tried to logout (thinking that I could relog with the good account, but I got lost on HP site without a possibility to come back on the forum and to register...

[Massimo Bianchi]

Hi,
just for your itrc login problem: delete the cookies relating to the irc and www5.itrc sites ...

So next time you can choose your account.

And for the forum site:

http://itrc.hp.com


Massimo