topic Re: Check return code via FTP in Operating System - HP-UX

Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 14:40:01 GMT

Here's my current FTP script that I run from another script via a cron job every night.

cd /outgoing
ftp -n -v > logfile.log <<-EOF
open 127.0.0.1
user user mypassword
ascii
put $1 'FTPFILE'
bye
EOF

I would like to add a check to this so that if the put fails somthing else happens. If we get a return code of anythin not = to 0 then we want to do the following. Can this be done within the FTP or does another FTP have to be done? Need some help on this. PLease mention exact steps. Thanks.

ascii
put blankfile 'BLANKFILE'
quit

Re: Check return code via FTP

A. Clay Stephenson — Thu, 12 Feb 2004 14:49:36 GMT

Actually, FTP uses 2 as a good return. By far, the better method is to use Perl's Net::FTP module; you get error checking for free.

Try this Perl script:

ftpput.pl myfile1 myfile2
STAT=${?}
if [[ ${STAT} -eq 0 ]]
then
echo "All is well"
fi

It will login (change the host, user, and password to suit your needs) and then set ascii mode, next it will put the files passed on the command line and actually resend on failure (up to 5 times).

Re: Check return code via FTP

Dave La Mar — Thu, 12 Feb 2004 15:01:14 GMT

Samuel -
Actually, since you were smart enough to create the log file during the ftp process, your error checking can go against the log file.
We commonly do as you have done and check the log file for may different conditions, i.e. byte counts, not connected messages, etc.
You get the idea.

Best of luck.

Regards,

dl

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 15:05:15 GMT

OK but I would have to do a separate FTP to implement the method of checking the logfile. Would I just have to do a grep on the file or something? I'd prefer to just use the FTP command directly.

Re: Check return code via FTP

Steven E. Protter — Thu, 12 Feb 2004 15:14:50 GMT

I have run a test.

ran a script that failed on a put but was able to login.

$? was still 0

Thats probably because the login didn't fail.

Even after a failed login the error code was zero.

A look at the help commands after an interactive ftp login reveals nothing that would seem to be helful

After the transfer though you can do this:

ls -la $1

At least then you'll get output to stdio that you can route to a file and process for errors later.

SEP

Re: Check return code via FTP

Steven E. Protter — Thu, 12 Feb 2004 15:18:16 GMT

While our Internet Connection was down I did some further research.

If you were transferring the file say via scp command line, you could check return codes after the transfer.

scp thisfile targethost://target_directory
rc=$?

if [ $rc -ne 0 ]
then
echo "Complain ... transfer failed"
fi

I have however noticed even on good transfers to a HP-9000 box having random seed problems that the return code has not always been zero.

I assume that once I resolve that issue my methodology will be reliable.

SEP

Re: Check return code via FTP

A. Clay Stephenson — Thu, 12 Feb 2004 15:21:03 GMT

You need to do a man ftpd for an explanation of replies sent to the ftp client upon receipt of commands. The actual replies considt of 3 digits but in almost all cases, all you need to do is examine the first (leftmost) digit and if it is a '2' then all is well. NOTE: This has NOTHING to do with the exit status of ftp.

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 15:27:24 GMT

Ok I'm FTPing FROM HP to an MVS. So the return codes may not be the same on both. Looks like this is pretty tough to do.

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 15:55:05 GMT

OK looks like unless I use Perl then there's no way "inside" the FTP script to do error handling.

Re: Check return code via FTP

Dave La Mar — Thu, 12 Feb 2004 15:58:52 GMT

Samuel -
We also ftp to and from a mainframe, but may of the error conditions are the same. In the ftp script, following the EOF, grep your log file for failure conditions or successful conditions.
If the grep is for failure, a RC=0 would idicate that particular failure was found. Using an if [ $RC = 0 ]
then
ux2dos logfile.log | uuencode logfile.txt | mailx -m -s "Failure in FTP" someone@somewhere.com
fi
That will email the log file as an attachment and notify an interested party as to the failure. The interested party will see the cause for failure in the log file.

On critical jobs we notify our operations department as well as others. If a call is necessary, ops calls the interested party.
In some cases we actually echo recoverin instruction to a file and then include it in the email as the body, i.e. (after the someon@somewhere.com) < recovery_instructions.file

I really believe you have all you need to perform what you desire.

Best regards,
dl

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 16:14:10 GMT

OK I'll go back to my original email.

"PLease mention exact steps. Thanks."

This is what we want to do in response to a failure.

"ascii
put blankfile 'BLANKFILE'
quit"

Looks like the above need to be a separate FTP that needs to be run based on the result of a grep of the log. I was thinking of this before but I would have preferred to keep only the one FTP script. I see why someone mentioned Perl. I have no experience with PErl so that's not an option for this rigr now.

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 16:30:17 GMT

OK so how would my if look if I was seeing if I got any results back from the grep. I'm going to search for a certain sequence in a line.

Re: Check return code via FTP

A. Clay Stephenson — Thu, 12 Feb 2004 16:41:24 GMT

I have a really strange question for you. If the put of the real file failed, what makes you think that a put of the blankfile will succeed?

This is a no-brainer even if you know no Perl because the changes are so easy.

1) On your HP-UX box, perl -v.
If that comes back with a version 5.6.1 or greater you are good to go as Net::FTP will be loaded for you otherwise should downlaod and install Perl from http://hpux.connect.org.uk/hppd/hpux/Languages/perl-5.8.3/.

2) I would now create a soft link for Perl in /usr/bin.
e.g. ln -s /opt/perl5/bin/perl /usr/bin/perl

3) The changes to the previously posted script are:

my $ftp = Net::FTP->new("remotehost",Debug => 0);
to

my $ftp = Net::FTP->new("127.0.0.1",Debug => 0);

------------------------------------
$ftp->login("cstephen","topsecret");
to
$ftp->login("user","mypassword");

and that's it.

If you insist upon trying to put the "blankfile" that is left as a dubious exercise. You could, of course, look at the exit status of the perl script and invoke a 2nd ftpput.pl blankfile and examine ${?} to see if that transmission was successful.

Re: Check return code via FTP

Dave La Mar — Thu, 12 Feb 2004 16:47:45 GMT

Here is a snip of the error checking in one of our ftp scripts.

grep 'Not connected' $FTP_LOG 1>/dev/null 2>/dev/null
echo $? | read RC
grep 'full' $FTP_LOG 1>/dev/null 2>/dev/null
echo $? | read RC1
grep 'fail' $FTP_LOG 1>/dev/null 2>/dev/null
echo $? | read RC2
grep 'Error writing' $FTP_LOG 1>/dev/null 2>/dev/null
echo $? | read RC3

if [ $RC -lt 1 -o $RC1 -lt 1 -o $RC2 -lt 1 -o $RC3 -lt 1 ]
then
echo "" >> $SCRIPT_HOME/messages
tail -5 $FTP_LOG >> $SCRIPT_HOME/messages
echo "" >> $SCRIPT_HOME/messages
echo "Problem with the CMSSVR server." >> $SCRIPT_HOME/messages
echo "" >> $SCRIPT_HOME/messages
echo "Gottschalks Operations ==> Review recovery procedures.">> $SCRIPT_HOME/messages
mailx -s "Nightly FTP failed for $SCRIPT_HOME" $MAIL_LIST < $SCRIPT_HOME/messages
exit

1. Of course grep -e may be used, but, in this case, I was interested in
the particular failure.
2. $MAIL_LIST is a variable (as may addresses as you'd like) like => export MAIL_LIST="someone@somewhere.com someone.else@somewhere.com"
3. $FTP_LOG is your ftplog.log
4. $SCRIPT_HOME is the path to your script directory.
5. tail -5 $FTP_LOG is generally all the error messages from the log file you will need in the body of the email to determine what happened.
6. In this example, there is no attachment as all is included in the email body.

Customize as you see fit.
Hope this helps.

Best regards,

dl

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 16:48:50 GMT

The reason for the blank file would work is if the dataset on the MVS side ran out of space. The first file would fail partially but the blank file would work and therefore the other side would know, because of the presence of the blank file, that the first file was incomplete.

Yeah I got 5.6.1.

Re: Check return code via FTP

A. Clay Stephenson — Thu, 12 Feb 2004 17:06:36 GMT

But iff (iff and only if) the receiver ran out of space. The number of reasons for FTP failures is legion.

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 17:09:26 GMT

Correct, but if the receiver doesn't receive the file that's ok. The problem we are trying to avoid is the reciever thinking they have received the complete file. They came up with the idea of us sending a blank file for them to check for.

If the FTP is successful I want to send another file. If it isn't, then I don't want to. Simple.

Re: Check return code via FTP

Samuel_10 — Thu, 12 Feb 2004 17:19:14 GMT

So A. Clay,

OK so it seems the Perl method would be the "cleanest" maybe. However it looks like I would end up modifying the default script you sent to include another FTP to maintain that cleaness.

IF I use your method that means another FTP, which is basically like going with a GRREP method and based on the results doing another FTP.

Re: Check return code via FTP

A. Clay Stephenson — Thu, 12 Feb 2004 17:30:27 GMT

Not really because you could put the check in the same Perl script. It took me under a minute to make that change and I reduced the retries to only 2.

Re: Check return code via FTP

A. Clay Stephenson — Thu, 12 Feb 2004 17:31:41 GMT

I'm an idiot, I missed the attachment.