LOGGING OUTPUT

[rmueller58]

I have several scripts that run in cron, I need to output information from the running of these scripts into logfiles.

I am curious what others are doing as best practice to log information?

My initial thought was to use the command in cron that sends stdout/stderr to the log by appending a redirection in the cron.

IE:
* * * * * /usr/local/bin/scriptname > /tmp/logname.log 2>&1

Or would it be better to log output from within the script itself ?

IE:
#!/bin/sh
LOG_FILE=/tmp/logname.log

command |tee $LOG_FILE 2>&1

One of the scripts is as follows by default,

crontab:
15 20 * * * /usr/local/bin/TLC_xfer.sh



#!/bin/sh
rm -f /tmp/staff_RAL.csv
rm -f /tmp/staff_WST.csv

export INFORMIXSERVER=pei
. /spi/spitools/bin/spi_prof.sh

/usr/informix/bin/isql wstplus /usr/informix/bin/isql wstplus /usr/informix/bin/isql ralplus
cat /usr/local/bin/ral_tlc_header /tmp/staff_RAL.csv > /tmp/staff_patron_download_RAL.csv
cat /usr/local/bin/wst_tlc_header /tmp/staff_WST.csv /tmp/staff-dc_WST.csv > /tmp/staff_patron_download_WST.csv


IF I modify the crontab What would be the command?

If I modify the script to log to /tmp/tlcmove.log


What would be best practice to output stdout/stderr to $LOG_FIL?


"TLC_xfer.sh" 13 lines, 523 characters

#!/bin/sh
LOG_FIL=/tmp/tlc_move.log

(NEED TO EDIT FOR ECHO TO LOG_FIL>output..)
rm -f /tmp/staff_RAL.csv
rm -f /tmp/staff_WST.csv

export INFORMIXSERVER=pei
. /spi/spitools/bin/spi_prof.sh

/usr/informix/bin/isql wstplus /usr/informix/bin/isql wstplus /usr/informix/bin/isql ralplus
cat /usr/local/bin/ral_tlc_header /tmp/staff_RAL.csv > /tmp/staff_patron_download_RAL.csv
cat /usr/local/bin/wst_tlc_header /tmp/staff_WST.csv /tmp/staff-dc_WST.csv > /tmp/staff_patron_download_WST.csv

[James R. Ferguson · Accepted solution]

Hi:

My usual preference is to put the controls in the script like:

#!/usr/bin/sh
typeset LOG=/var/tmp/$(basename $0).$$
[ -t 0 ] || exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."

...now, if I run this interactively, the output goes to my terminal unless I explicitly redirect it. If I use this in a 'crontask' or 'at' job, the output is automatically redirected to the LOG file. The choice is really entirely up to you, though. Providing redirection in 'cron' is wholly appropriate, too.

As for temporary file cleanup, I like to do:

#!/usr/bin/sh
typeset TMPFILE="/var/tmp/$(basename $0).$$
trap 'rm -f ${TMPFILE}' EXIT

This names the TMPFILE with the 'basename' of the running script suffixed with the current pid. Upon exit (from anywhere in the script) the "epilog" remove of the file occurs.

Regards!

...JRF...

[rmueller58]

James,

using the "code" you suggested
#!/usr/bin/sh
typeset LOG=/var/tmp/$(basename $0).$$
[ -t 0 ] || exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."


will this log all output from the rest of the script without using redirection in the command line?

would use a "tee" I do?

IE:
#!/usr/bin/sh
typeset LOG=/var/tmp/$(basename $0).$$
[ -t 0 ] || exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."

rm -f /tmp/staff_RAL.csv |tee ${LOG)
rm -f /tmp/staff_WST.csv |tee ${LOG)

... (rest of script truncated)

#END

Or not use a "tee"
OR

#!/usr/bin/sh
typeset LOG=/var/tmp/$(basename $0).$$
[ -t 0 ] || exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."


rm -f /tmp/staff_RAL.csv
rm -f /tmp/staff_WST.csv

# TRUNCATED.....

[James R. Ferguson]

Hi (again):

> using the "code" you suggested...will this log all output from the rest of the script without using redirection in the command line?

This will log all STDOUT and STDERR output for the duration of the script into the file "${LOG}". _HOWEVER_ this only occurs if the process is _NOT_ associated with a terminal. That's what the '-t 0' tests. It says if STDIN isn't a terminal, redirect STDOUT and STDIN.

> would use a "tee" I do?

Since you are using the '-f' switch with your 'rm' command, there isn't going to be any STDERR output for non-existent files. Hence there is no need to use 'tee' or to redirect STDERR.

Regards!

...JRF...

[rmueller58]

the rm -f was a bad example

how about if i do something like:

ls -la /tmp/staff*csv

I would need to redirect through tee ${LOG)?

[rmueller58]

Disregard last post I am a bit clogged with a cold and didn't see that it logs all from the script

[James R. Ferguson]

Hi (again):

Using 'tee' and having 'exec'd STDOUT to a file are mutually exclusive. Once you have performed the 'exec' redirection, the output descriptors are set.

Regards!

...JRF...

[rmueller58]

Thanks James, this is going to clean up a lot of stuff for me..

Kudos.. Wish I could assign more the 10 points per..

[James R. Ferguson]

Hi (again):

Actually, there are some other useful things we can do by duplicating file descriptors. Consider this:

# cat ./showme
#!/usr/bin/sh
typeset LOG=/var/tmp/$(basename $0).$$
exec 4>&1 #...duplicate STDOUT to FD-4
exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."
print -u4 "...but this stays on my terminal!..."
ls -l >&4 #...and this goes to the terminal ALSO...
exit

...

Regards!

...JRF...

[rmueller58]

James,
I am assuming if I put the LOGGING mechanism above the commands all items if ran in cron should append to $LOG

and if command name is TLC_xfer.sh the LOG name should be similar..

What I have now is as follows:
# more TLC_xfer.sh
#!/bin/sh
typeset LOG=/tmp/$(basename $0).$$
[ -t 0 ] || exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."
ls -l >&4 "#send ls command to Term ..."
exit

### TRUNCATE RUNNING ITEMS ###
/usr/informix/bin/isql ralplus
cat /usr/local/bin/ral_tlc_header /tmp/staff_RAL.csv > /tmp/staff_patron_download_RAL.csv

Rex

[James R. Ferguson]

HI (again) Rex:

> and if command name is TLC_xfer.sh the LOG name should be similar

Yes, the '$(basename $0)' gets the basename of the process.

Actually, putting everything together we can use accomodate your 'tee'. Consider:

# ./mysh
#!/usr/bin/sh
set -u
typeset LOG=/var/tmp/$(basename $0)
typeset TMP=/var/tmp/$(basename $0).$$
trap 'rm -f ${TMP}' EXIT
exec 4>&1 #...duplicate STDOUT to FD-4
exec > ${LOG} 2>&1
echo "this is STDOUT..."
print -u2 "this is STDERR..."
ls -l > ${TMP}
cat ${TMP} >&4
cat ${TMP} >> ${LOG}
exit

Regards!

...JRF...

[rmueller58]

James, Thanks.. I am going to run it through cron tonight and debug if anything comes up.. Thanks again.. Rex