How to find out if the FTP file is finished loading

[Jade Bulante]

I would like to write a program that would move files after finishing FTP process but I need to find out if the file has finished downloading before doing it. Is there any way to do this? How about fuser? Any ideas?

[Sridhar Bhaskarla · Accepted solution]

Hi Jade,

'fuser' is a good idea but it is not available for normal users.

If you are 'get'ting the file, then you will have to write the script after that session is completed.

If you are putting the file from somewhere, then the local 'ftpd' will be active on the system. You will see a string something like 'STOR ' associated with ftpd process. You can search for it before you move the files.

-Sri

[A. Clay Stephenson]

As long as your are putting, the mv will be safe because as long as ftp has the file open it will not be unlinked BUT you have zero error checking with your present scheme. The file may no longer be in use (so lsof or fuser is happy) but the transmission may have quit before completion and you don't know it. It's much smarter to put some error checking in the FTP transfer itself. When the command completes, you know its okay because you can check the error status.

I would make use of Perl's Net::FTP module; the good news is that I already have a Perl script that you can simply call from the shell to do all the hard work.

Use it like this:

REMOTE_HOST=mickey
REMOTE_DIR=/u01/mydir
REMOTE_USER=cstephen
PASSWD=top_secret
RETRIES=3



ftpput.pl -h ${REMOTE_HOST} -l ${REMOTE_DIR} -p ${PASSWD} -d ${REMOTE_DIR} -t ${RETRIES} myfile1 myfile2 myfile3
STAT=${?}

In this case, if ${STAT} -eq 0 then myfile1, myfile2, myfile3 were sent sucessfully and you can now move them.

Invoke as ftpput.pl -u for full usage. Note that you can omit the passwd if it is specified in the user's .netrc file (just like FTP).

[A. Clay Stephenson]

In case you need to "get" the files, here is ftpget.pl. Invoke as ftpget.pl for usage.

[Jade Bulante]

Actually, I'm the one getting the file and before I run my program, I need to know if the file transfer has been completed.

[A. Clay Stephenson]

The solution is all but identical, simply use the ftpget.pl script I attached earlier. If you see a zero result, you know that you got all the file intact. You might want to add the -B (default, binary transfer) or -A (ASCII transfers) argument to the request.

#!/usr/bin/sh
REMOTE_HOST=mickey
REMOTE_DIR=/u01/mydir
REMOTE_USER=cstephen
PASSWD=top_secret
RETRIES=3
BIN_OR_ASCII="-A" # ASCII xfer


ftpget.pl -h ${REMOTE_HOST} -l ${REMOTE_DIR} -p ${PASSWD} -d ${REMOTE_DIR} -t ${RETRIES} ${BIN_OR_ASCII} myfile1
STAT=${?}

This will log you in on the remotehost, cd to the indicated directory, assert ASSCI mode, and try to get myfile up to 3 times before giving up. If ${STAT} is zero, you got all the file. Invoke as ftpget.pl for full usage. Again, the passwd can be read from the .netrc file if available.

[A. Clay Stephenson]

Oops, I'm an idiot.

That should have been "-l ${REMOTE_USER}" rather than "-l ${REMOTE_DIR}" in all cases above.

ftpget.pl -h ${REMOTE_HOST} -l ${REMOTE_USER} -p ${PASSWD} -d ${REMOTE_DIR} -t ${RETRIES} ${BIN_OR_ASCII} myfile1
STAT=${?}