System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

How to setup an email alert notification for failed SFTP jobs?

SOLVED
Go to solution
Jan Shu
Regular Advisor

How to setup an email alert notification for failed SFTP jobs?

Hi,

My DBA setup a sftp cron job to upload data to a remote sftp server. How to setup an email alert notification for failed SFTP jobs? so my DBA can resend the data transfer. Can you please help? Thank you.

Regards,
Jan
23 REPLIES
James R. Ferguson
Acclaimed Contributor
Solution

Re: How to setup an email alert notification for failed SFTP jobs?

Hi Jan:

If you SFTP script returns zero for successs and non-zero for failure, then you simply do something like:

# /home/jan/script || mailx -s "Failure in script" root < /dev/null

Regards!

...JRF...
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Dear James,
Thanks for the reply. I asked my DBA if her sftp script returns zero and non-zero for job status. DBA isn't sure, but she shows me the sftp log, it has only the verbes of the sftp connection. I think by default a script always return zero or non-zero for its result, right? Is there a way to test it?

Thanks.

Regards,
Jan
OldSchool
Honored Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

sftp, like ftp, appears to always return a zero, provided there isn't an error in sftp itself.

testing via the command line, and "echo $?" will show 0 regardless, you can prove this by trying to copy an non-existent file on the command line. the "script" will have to be written such that it scans to output log or takes some other action to confirm the file transfered completely / successfully. based on that action, the script should then return a zero or non-zero exit code.....
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Dear OldSchool,

Thanks for the information. I will let my DBA test this. Please wait for my update soon. Thanks again.

Regards,
Jan
James R. Ferguson
Acclaimed Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi (again):

As OldSchool notes, the return value of (S)FTP isn't going to help. You need to capture the session dialog in a file and parse it for success or failure.

If, however, your (S)FTP script were written in Perl, the Perl module implementations allow simple tests like:

...
$sftp->put( $localfile, $remotefile );
if ( $sftp->error ) {
warn "Can't put '$localfile' -> '$remotefile': " . $sftp->error . "\n";
...

Regards!

...JRF...
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Dear James,

I will check with my DBA if their sftp script is in Perl or not. I will search this forum website and hopefully I can find some sample sftp scripts in this forum website.

Thanks again.

Jan
Steven E. Protter
Exalted Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

Shalom,

You could go as simple as this:

If the sftp job fails send an email.

If its a shell script.

sftp or scp command ...
rc=$?
#This gets you the return code.

if [ $rc -ne 0 ]
then
echo "File transfer error" | mailx -s "subject" email@yuor.net
fi

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Dear Stephen,

Thanks a lot, I will try this too. Please wait for my update.

Have a nice day.

Regards,
Jan
James R. Ferguson
Acclaimed Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi (again) Jan:

Re-read OldShool's post. A simple return code from SFTP in a shell script is _not_ going to work:

Consider:

# cat ./mysh
#!/usr/bin/sh
sftp somehost << EOF!
get /no/file /tmp/nofile
quit
EOF!
RC=$?
echo "rc=${RC}"

# ./mysh
Connecting to somehost...
sftp> get /no/file /tmp/nofile
Couldn't stat remote file: No such file or directory
File "/no/file" not found.
sftp> quit
rc=0

Regards!

...JRF...
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi James,

My DBA says their sftp script is a shell script not in Perl. If the return code won't work in the sftp shell script and I don't want to ask DBA to rewrite the script in Perl, what else should I try?

Thanks,
Jan
OldSchool
Honored Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

as noted previously:

"the "script" will have to be written such that it scans to output log or takes some other action to confirm the file transfered completely / successfully. based on that action, the script should then return a zero or non-zero exit code....."

In other words: somebody's going to have to write / rewrite something to make this work.
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi James,

OldSchool wrote: "sftp, like ftp, appears to always return a zero, PROVIDED there isn't an error in sftp itself.
" So if there is an error, then it will return non-zero. And it should work for my sftp shell script to send failure alert notification.

Jan
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Just got OldSchool's new update. Thanks.
OldSchool
Honored Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

"So if there is an error, then it will return non-zero. And it should work for my sftp shell script to send failure alert notification"

No..No.NO!

sftp ALWAYS returns a zero, even if the commands fed to it fail. the only time you would get a non-zero is if something caused "sftp" itself to fail, like no space in process table.

Read and understand JRF's example. The transfer *FAILED* because of missing file, but the return code was 0 (indicating success)
James R. Ferguson
Acclaimed Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi (again) Jan:

Here's a quick example of how you _might_ parse the SFTP session to determine success or failure:

# cat ./mysftp
#!/usr/bin/sh
typeset RC
typeset SFTPLOG=/tmp/sftplog.$$
trap 'rm ${SFTPLOG}' EXIT
exec 2> ${SFTPLOG}
sftp smhadt1 << EOF!
get /no/file /tmp/nofile
quit
EOF!
grep -iq "couldn't stat" ${SFTPLOG} && RC=1 || RC=0
echo "rc=${RC}"
exit ${RC}

...customize to your needs...

Regards!

...JRF...
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi James,

Thanks. Is this example a Perl script? Will it work with my DBA's shell script? e.g. I copy/paste the example commands to my DBA's script on a HPUX 11.23 Oracle 8i server?

Regards,
Jan
James R. Ferguson
Acclaimed Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi (again) Jan:

> Is this example a Perl script?

No, it is a SHELL script. The '#!/usr/bin/sh' (first) line signals what interpreter to use --- the HP-UX Posix shell in this case.

> Will it work with my DBA's shell script?

You can certainly integregate and embellish the concept I presented. I would make the hostname, the source and the target file names arguments to pass, too.

Regards!

...JRF...
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Thanks, James. I will try this, please wait.

Jan
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi James/OldSchool/SEP,

I got an idea. How about comparing the check sum of the data files before and after the sftp transfer? If they are the same, then the sftp upload is OK, if the check sum is different, then it is failed. What do you think?

Thanks.

Jan
James R. Ferguson
Acclaimed Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi (again) Jan:

> I got an idea. How about comparing the check sum of the data files before and after the sftp transfer? If they are the same, then the sftp upload is OK, if the check sum is different, then it is failed. What do you think?

That's OK _as_long_as_ your not moving a text file from UNIX to WINDOWS or vice versa using the ASCII mode of transfer. When you do that, a carriage return character will helpfully be added or subtracted from each "line" rendering the checksums different.

Regards!

...JRF...
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Hi James,

Thanks for the reply. The sftp server is a UX server (not sure Sun or HPUX) and I will make sure we use binary mode to upload. Thanks again.

Jan
OldSchool
Honored Contributor

Re: How to setup an email alert notification for failed SFTP jobs?

JRF said: "That's OK _as_long_as_ your not moving a text file from UNIX to WINDOWS or vice versa using the ASCII mode of transfer. When you do that, a carriage return character will helpfully be added or subtracted from each "line" rendering the checksums different."


while correct for ascii mode transfers in conventional FTP, some of the SFTP stuff, especially early open-source, don't make to dos2ux / ux2dos conversions. I won't go into how I happen to know about this ;)
Jan Shu
Regular Advisor

Re: How to setup an email alert notification for failed SFTP jobs?

Thank you all for your helpful inputs on this.

Jan