Script Problem

[Ali Imran Abbas]

I have a script here on hp-ux server that backups oracle database. This script is written by some one else, the problem is with this script is that after it is run and backup is done, it leaves the process stalled on the system. It doesnt kill the process after completng the job.
One solution to it is to grep the process id and then kill it and get this job done by cron, but my manager doesnt want that. He wants to know why the script is not finishing successfully and why it is leaving process stalled on the system. Does any one knows how to fix this problem?

[Patrick Wallek]

If the script is "stalled", then it is more than likely waiting on something.

Does the script generate a log file? If so, does that provide any clue?

Is this a shell script? If so, can you run it with the '-x' option to enable some debugging?

Is it possible to post the script? Without the script and/or some more information about what it is, or isn't, doing it will be difficult to help much.

[Robert Salter]

Well, it might help to post the script. Or you could look to see if there was any exit in the script. What utility was used for the backups? tar, fbackup, ??

Need more information in order to proffer any useful help.

ciao

[Ali Imran Abbas]

We are using data protector 5.5. Here is the script:

#!/usr/bin/ksh
#===================================================================#
# Name dbbackup.scr #
# #
# Purpose Parses the backup schedule file and calls the #
# appropriate backup script. The three backup #
# scripts that get called are: #
# 1. Backup (dbbackup_begin.scr) #
# 2. Export (dbexport_begin.scr) #
# 3. Special (dbspecial_begin.scr) #
# #
# This script also calls the dbbackup.env script to #
# set environment variables for the passed in dbname. #
# #
# The PATH in the dbbackup.env file must contain the #
# following UNIX commands: #
# awk date kill sed #
# basename df ls sleep #
# cat echo mailx sort #
# chmod egrep mknod tail #
# compress find mv wall #
# cp grep ps wc #
# cut head rm #
# #
# Usage dbbackup.scr #
# #
# Parameters $1=dbname #
# #
# Modifications #
# #
# Date Person Description of Modification #
# -------- --------------- ------------------------------------- #
# #
#===================================================================#
#===========================================================#
# VARIABLES #
#===========================================================#
NUMLINES=0
ERRMSG="Usage: `basename $0` "
DBA_WORKS=/opt/app/oracle/admin/DBA_WORKS;export DBA_WORKS
DBA_ADMIN=/opt/app/oracle/admin;export DBA_ADMIN
BACKUPENV="$DBA_WORKS/common/crontab.env"

if [ "$1" ]; then
DBNAME=$1
else
echo $ERRMSG
exit
fi

#===========================================================#
# SET ENVIRONMENT FOR PASSED IN #
# #
# This should define ORACLE_HOME, ORACLE_SID, PATH, #
# BACKUPDIR, BACKUPDIROLD, EXPORTDIR, EXPORTDIROLD, ARCOLD #
#===========================================================#
. $BACKUPENV ${DBNAME}

#===========================================================#
# SET BACKUP SCHEDULE FILE & DEFINE BACKUP SCRIPTS #
# #
# SCHED_FILE is the file containing backup schedule. #
# A sample of the schedule file is as follows: #
# #
# qapp Mon cold export /u05/backup/special.scr #
# #
# JOBNAME is the current script. #
# #
# DBBACKUP_BEGIN is the backup script that performs hot #
# or cold backup based on the schedule of the day. #
# #
# DBEXPORT_BEGIN is the script for exporting the database. #
# #
# DBSPECIAL_BEGIN is the script for running special scripts #
# after the backup and export has occured. #
#===========================================================#
THIS_DAY="`date +'%a'`"
MSG="$DBNAME Backup succeeded at `date`"

# ..............................................................
# begin
# ..............................................................
if [ -f "$ERRFILE" ]; then
if [ ! -s $ERRFILE ]; then # Delete .err file if empty
rm ${ERRFILE}
else
cat $ERRFILE >> $ERRFILE2 # Save old .err file
fi
fi

#
# Read backup schedule
#
awk -v dbname=$DBNAME -v this_day=$THIS_DAY '{
#
# get a record
#
cmd=$0
sizeofarray=split(cmd,rec," ")
dbname2=rec[1]
day_of_week=rec[2]
backup=rec[3]
export=rec[4]
special_task=rec[5]

if (( dbname2 == dbname ) && ( this_day == day_of_week ))
print " " backup " " export " " special_task
}' $SCHED_FILE | while read BACKUP EXPORT SPECIAL_TASK
do
#
# Print all parameters before beginning backup.
#
echo ".................................................................."\
>> $LOGFILE 2> $ERRFILE
echo " BACKUP JOB PARAMETERS:" >> $LOGFILE 2> $ERRFILE
echo " " >> $LOGFILE 2> $ERRFILE
echo " Host Name: `uname -n` " >> $LOGFILE 2> $ERRFILE
echo " Database Name: $DBNAME" >> $LOGFILE 2> $ERRFILE
echo " Backup Type: $BACKUP" >> $LOGFILE 2> $ERRFILE
echo " Export Type: $EXPORT" >> $LOGFILE 2> $ERRFILE
echo " Special Task: $SPECIAL_TASK" >> $LOGFILE 2> $ERRFILE
echo " ..............................................."\
>> $LOGFILE 2> $ERRFILE
echo " ENVIRONMENT VARIABLES:" >> $LOGFILE 2> $ERRFILE
echo " " >> $LOGFILE 2> $ERRFILE
echo " ORACLE_SID: $ORACLE_SID" >> $LOGFILE 2> $ERRFILE
echo " ORACLE_HOME: $ORACLE_HOME" >> $LOGFILE 2> $ERRFILE
echo " PATH: $PATH" >> $LOGFILE 2> $ERRFILE
echo " ..............................................."\
>> $LOGFILE 2> $ERRFILE
echo " BACKUP DIRECTORIES:" >> $LOGFILE 2> $ERRFILE
echo " " >> $LOGFILE 2> $ERRFILE
echo " ARCDIR: $ARCDIR" >> $LOGFILE 2> $ERRFILE
echo " BACKUPARC: $BACKUPARC" >> $LOGFILE 2> $ERRFILE
echo " BACKUPDIR: $BACKUPDIR" >> $LOGFILE 2> $ERRFILE
echo " BACKUPARCOLD: $BACKUPARCOLD" >> $LOGFILE 2> $ERRFILE
echo " BACKUPDIROLD: $BACKUPDIROLD" >> $LOGFILE 2> $ERRFILE
echo " EXPORTDIR: $EXPORTDIR" >> $LOGFILE 2> $ERRFILE
echo " EXPORTDIROLD: $EXPORTDIROLD" >> $LOGFILE 2> $ERRFILE
echo " " >> $LOGFILE 2> $ERRFILE
echo " /~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\\" \
>> $LOGFILE 2> $ERRFILE
echo " | DISK USAGE REPORT |" \
>> $LOGFILE 2> $ERRFILE
echo " |_________________________________________________________|" \
>> $LOGFILE 2> $ERRFILE
echo " | Mounted On | Size(MB)| Free(MB)| %Used|" \
>> $LOGFILE 2> $ERRFILE
echo " |------------------------------|---------|---------|------|" \
>> $LOGFILE 2> $ERRFILE
LOGB=`echo $LOGDIR|cut -d "/" -f2`
EDOB=`echo $EXPORTDIROLD|cut -d "/" -f2`
EDB=`echo $EXPORTDIR|cut -d "/" -f2`
BDB=`echo $BACKUPDIR|cut -d "/" -f2`
BDOB=`echo $BACKUPDIROLD|cut -d "/" -f2`
ADB=`echo $ARCDIR|cut -d "/" -f2`
BAOB=`echo $BACKUPARCOLD|cut -d "/" -f2`
df -b| awk '{printf " |%30s|%9.2f|%9.2f|%5s |\n",$1,($2/1024),($5/1024),$4}'
\
| grep -e "$LOGB" -e "$EDOB" -e "$EDB" -e "$BDB" \
-e "$BDOB" -e "$ADB" -e "$BAOB" >> $LOGFILE 2> $ERRFILE
echo " \\~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~/" \
>> $LOGFILE 2> $ERRFILE
echo ".................................................................."\
>> $LOGFILE 2> $ERRFILE

#
# Cleanup Old Log Files
#
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "DELETING OLD LOG FILES " >> $LOGFILE 2>> $ERRFILE
OLDLOGCOUNT=0
for OLDFILE in `find ${LOGDIR} -name "*" -mtime +$KEEPLOGDAYS`
do
rm ${OLDFILE}
(( OLDLOGCOUNT = OLDLOGCOUNT + 1 ))
done
echo "Deleted ${OLDLOGCOUNT} files > ${KEEPLOGDAYS} Days Old." \
>> $LOGFILE 2>> $ERRFILE
echo " " >> $LOGFILE 2>> $ERRFILE

#
# Backup
#
if [ "$BACKUP" != "nobackup" ]; then
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "Begin backup at `date +'[%m/%d/%Y %I:%M:%S%p]'`"\
>> $LOGFILE 2>> $ERRFILE
echo "..................................." >> $LOGFILE 2>> $ERRFILE
$DBBACKUP_BEGIN $DBNAME $BACKUP >> $LOGFILE 2>> $ERRFILE
echo "$DBBACKUP_BEGIN $DBNAME $BACKUP >> $LOGFILE 2>> $ERRFILE"
echo
echo
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "End backup at `date +'[%m/%d/%Y %I:%M:%S%p]'`" \
>> $LOGFILE 2>> $ERRFILE
echo "..................................." >> $LOGFILE 2>> $ERRFILE
fi

#
# Export
#
if [ "$EXPORT" != "noexport" ]; then
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "Begin export at `date +'[%m/%d/%Y %I:%M:%S%p]'`" \
>> $LOGFILE 2>> $ERRFILE
echo "..................................." >> $LOGFILE 2>> $ERRFILE
$DBEXPORT_BEGIN $DBNAME $EXPORT >> $LOGFILE 2>/dev/null
echo "$DBEXPORT_BEGIN $DBNAME $EXPORT >> $LOGFILE 2>> $ERRFILE"
echo
echo
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "End export at `date +'[%m/%d/%Y %I:%M:%S%p]'`" \
>> $LOGFILE 2>> $ERRFILE
echo "..................................." >> $LOGFILE 2>> $ERRFILE
fi

#
# Special Task
#
if [ "$SPECIAL_TASK" ]; then
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "Begin Special Task at `date +'[%m/%d/%Y %I:%M:%S%p]'`" \
>> $LOGFILE 2>> $ERRFILE
echo "..................................." >> $LOGFILE 2>> $ERRFILE
$DBSPECIAL_BEGIN $DBNAME $SPECIAL_TASK >> $LOGFILE 2>> $ERRFILE
echo "$DBSPECIAL_BEGIN $DBNAME $SPECIAL_TASK >> $LOGFILE 2>> $ERRFILE"
echo
echo
echo "..................................." >> $LOGFILE 2>> $ERRFILE
echo "End Special Task at `date +'[%m/%d/%Y %I:%M:%S%p]'`" \
>> $LOGFILE 2>> $ERRFILE
echo "..................................." >> $LOGFILE 2>> $ERRFILE
fi

ERRCNT=`egrep -e error -e ORA- -e EXP- -e fatal $LOGFILE | \
grep -c -v "No errors."`
ERRCNT2=`egrep -e error -e ORA- -e EXP- -e fatal $ERRFILE | \
grep -c -v "Export terminated successfully"`


if [ "$ERRCNT" -gt 0 -o "$ERRCNT2" -gt 0 ]; then
MSG="$DBNAME backup failed at `date + '[%m/%d/%Y %I:%M:%S%p]'`"
banner Pegasus System backup failed > ${LOGDIR}/tmp.dbbackup
date >> ${LOGDIR}/tmp.dbbackup
$EMAILERROR "${DBNAME}" "${MSG}" "${LOGFILE}" "${EMAILSUBJECT}" "${EMAILTOL
IST}"
else
banner Pegasus System backup successful > ${LOGDIR}/tmp.dbbackup
date >> ${LOGDIR}/tmp.dbbackup
MSG="$DBNAME backup succeeded at `date +'[%m/%d/%Y %I:%M:%S%p]'`"
$EMAILERROR "${DBNAME}" "${MSG}" "${LOGFILE}" "${EMAILSUBJECT}" "${EMAILTOL
IST}"
fi

#
# Check Disk Space for Log Files For Next Backup
#
LROOT=/`echo $LOGDIR | cut -d "/" -f2`
LOGSPACE=`df -k ${LOGDIR}|tail -1|awk '{print $1}'`
(( LOGSPACE = LOGSPACE * 1024 *1024))
if [ "$LOGSPACE" -lt "$MINLOGSPACE" ]; then
echo " WARNING < ${MINLOGSPACE} Disk Space For Log Files in ${LROOT}" \
>> $LOGFILE 2> $ERRFILE
$EMAILERROR "${DBNAME}" "[dbbackup.scr] - **WARNING** < ${MINLOGSPACE} Disk
Space in ${LROOT} For Log Files For NEXT BACKUP" "${LOGFILE}" "${EMAILSUBJECT}"
"${EMAILTOLIST}"
fi

if [ ! -s $ERRFILE ]; then
rm ${ERRFILE} # Delete .err file if empty
fi

done


And here is the other script that is called by the first script:

[Ali Imran Abbas]

the second script is:

#!/bin/ksh
#===================================================================#
# Name dbbackup_begin.scr #
# #
# Purpose Performs a hot or cold backup of the database. #
# This script is called by dbbackup. #
# #
# Usage dbbackup_begin.scr #
# #
# Parameters $1=dbname #
# $2=backup type ('hot','cold','archs') #
# #
# Modifications #
# #
# Date Person Description of Modification #
# -------- --------------- ------------------------------------- #
# #
#===================================================================#
#===========================================================#
# FUNCTIONS #
#===========================================================#
#===================================================#
# Name copy_and_compress #
# #
# Purpose This function copies and compresses #
# the SourceFile to the TargetFile in #
# one statement by using a pipe. #
# #
# Parameters 1 - SourceFile #
# 2 - TargetFile #
# #
#===================================================#
copy_and_compress()
{
SOURCEFILE=$1
TARGETFILE=$2
PIPEFILE=/tmp/${DBNAME}_candc.pipe

if [ -p $PIPEFILE ]; then
rm ${PIPEFILE}
fi

mknod $PIPEFILE p
sleep 5
chmod 644 $SOURCEFILE
compress < ${PIPEFILE} > ${TARGETFILE}.Z &
sleep 5
cp ${SOURCEFILE} ${PIPEFILE}
chmod 640 $SOURCEFILE

if [ $? != 0 ]; then
echo " ERROR During Copy & Compress: ${SOURCEFILE}"
fi

sleep 10
if [ -p $PIPEFILE ]; then
rm ${PIPEFILE}
fi
} # End of copy_and_compress

#===================================================#
# Name cleanup_db_files #
# #
# Purpose This function removes all of the #
# parametric files used by this #
# script. #
# #
# Parameters None #
# #
#===================================================#
cleanup_dbfiles()
{
echo " .........................................................."
echo " CLEANING UP PARAMETRIC FILES "
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " "
if [ -f $TBLSPACE ]; then
echo " Removing ${TBLSPACE}"
rm ${TBLSPACE}
fi

if [ -f $DATAFILE ]; then
echo " Removing ${DATAFILE}"
rm $DATAFILE
fi

if [ -f $ALERTLOGS ]; then
echo " Removing ${ALERTLOGS}"
rm $ALERTLOGS
fi

if [ -f $DBFILE ]; then
echo " Removing ${DBFILE}"
rm ${DBFILE}
fi
} # End of cleanup_dbfiles()

#===================================================#
# Name check_database_status #
# #
# Purpose This function checks to see if the #
# database is ONLINE or OFFLINE. It #
# then sets the DBSTATUS variable #
# appropriately. #
# #
# Parameters None #
# #
#===================================================#
check_database_status()
{
NUMPROCS=`ps -ef | egrep ora | grep "ora_[a-z]*_${DBNAME}"|wc -l`

if [ $NUMPROCS = 0 ]; then # Database is Offline
DBSTATUS="OFFLINE"
else
DBSTATUS="ONLINE"
fi # End of Checking Database Online Status

} # End of check_database_status()

#===================================================#
# Name copy_archive_logs #
# #
# Purpose This function backs up the archive #
# log files. If a "REPLACE" is passed #
# in to the function, the old archive #
# logs currently in the archive #
# directory will be moved to the #
# archive_old directory. If an #
# "APPEND" is passed in to the #
# function, the archive log files #
# will be copied into the archive log #
# directory. #
# #
# Parameters 1 - Backup Type ("REPLACE","APPEND")#
# #
#===================================================#
copy_archive_logs()
{
# Local Variables
L_ARCHTYPE=$1

# Get Log File Directory and Force a log switch
$SQL_UTILITY mode=line > /dev/null < connect internal
alter system switch logfile;
exit;
EOF
# Sleep until the log file is copied.
sleep 10

if [ $L_ARCHTYPE = REPLACE ]; then
# Remove Old Archive Log Files
echo " .........................................................."
echo " REMOVING OLD ARCHIVE LOG FILES FROM ${BACKUPARCOLD}"
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " \$BACKUPARCOLD: ${BACKUPARCOLD} "
echo " "

if [ `ls ${BACKUPARCOLD} | wc -l` != 0 ]; then
for I in ${BACKUPARCOLD}/*
do

echo " Removing Archive Log File \$BACKUPARCOLD/`basename ${I}`"
rm ${I}
if [ $? != 0 ]; then
echo " ERROR Removing Archive Log \${BACKUPARCOLD}/`basenam"
fi
done
else
echo " No Old Archive Logs to Remove "
fi

# Move Archive Logs to Archive Old Directory
echo " .........................................................."
echo " MOVING ARCHIVE LOG FILES TO ${BACKUPARCOLD}"
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'` "
echo " \$BACKUPARCOLD: ${BACKUPARCOLD} "
echo " \$BACKUPARC: ${BACKUPARC} "
echo " "
"CJH Remove old archives"
if [ `ls ${BACKUPARC} | wc -l` != 0 ]; then
for I in ${BACKUPARC}/*
do
echo " Moving \$BACKUPARC/`basename $I` TO \$BACKUPARCOLD"
rm ${I} $BACKUPARCOLD

if [ $? != 0 ]; then
echo " ERROR Moving Archive Log \${BACKUPARC}/`basename $I`"
fi
done
else
echo " No Archive Log Files to Move to Old Directory"
fi
fi # End of If L_ARCTYPE="REPLACE"

# Copy Archive Logs to Archive Directory
echo " .........................................................."
echo " MOVING ARCHIVE LOG FILES TO ${BACKUPARC}"
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " \$BACKUPARC: ${BACKUPARC} "
echo " \$ARCDIR: ${ARCDIR} "
echo " "

if [ -d $ARCDIR ]; then # ARC Directory Exists
if [ `ls ${ARCDIR} | wc -l` != 0 ]; then
for I in ${ARCDIR}/*
do
ARCNAME=`basename $I`
FSIZE=`ls -lst ${I} | awk '{print $6}'`
echo " ........................................ "
echo " [${ARCNAME} (Size:${FSIZE}) - Started `date +'%I:%M%p'`]"
echo " Copying \$ARCDIR/${ARCNAME}"
copy_and_compress ${I} ${BACKUPARC}/${ARCNAME}

echo " Compressing \$BACKUPARC/${ARCNAME}"
echo " Removing \$ARCDIR/${ARCNAME}"
echo " "
rm ${ARCDIR}/${ARCNAME}
if [ $? != 0 ]; then
echo " Archive deletion failed"
fi
done
else
echo " Found no archives to copy"
fi # End of [ ls ${ARCDIR}... ]
else
echo " Archive Directory doesn't exist."
fi # End of [ -d $ARCDIR ]
} # END OF copy_archive_logs()

#===========================================================#
# MAIN PROGRAM #
# #
#===========================================================#
#===========================================================#
# CHECK PARAMETERS #
#===========================================================#

# Check Passed In Parameters
if [ "$1" ]; then
DBNAME=$1
else
echo $ERRMSG >&2
exit 1
fi

if [ "$2" ]; then
BACKUP=$2
else
echo $ERRMSG >&2
exit 1
fi

#
# Only run script if passed in a "hot","cold" or "archs" and
# special task is not blank
#
if [ $BACKUP != "hot" ]; then
if [ $BACKUP != "cold" ]; then
if [ $BACKUP != "archs" ]; then
exit
fi
fi
fi
#=============================================================================#
# LOCAL VARIABLES #
#=============================================================================#
BACKUPENV="/opt/app/oracle/admin/DBA_WORKS/common/crontab.env"
THISSCRIPT=`basename $0`
ERRMSG="Usage: ${THISSCRIPT} "
JOBNAME="dbbackup_begin" # Name of this script
DBSTATUS=" " # Either ONLINE or OFFLINE
TMPDIR="/tmp" # Temp Directory
DATAFILES="${TMPDIR}/${DBNAME}_dfile_${FDATE}.log" # DB datafiles list
DATAFILE="${TMPDIR}/${DBNAME}_dfile_*.log" # DB datafiles list
TBLSPACES="${TMPDIR}/${DBNAME}_tblspc_${FDATE}.log" # DB tablespaces list
TBLSPACE="${TMPDIR}/${DBNAME}_tblspc_*.log" # DB tablespaces list
ALERTLOG="${TMPDIR}/${DBNAME}_alert_${FDATE}.log" # DB Alert Log Dir
ALERTLOGS="${TMPDIR}/${DBNAME}_alert_*.log" # DB Alert Log Dir
DBFILES="${TMPDIR}/${DBNAME}_alldbfile_${FDATE}.log" # DB All datafiles list
DBFILE="${TMPDIR}/${DBNAME}_alldbfile_*.log" # DB All datafiles list
#=============================================================================#
#=============================================================================#
# SET BACKUP ENVIRONMENT #
#=============================================================================#

. $BACKUPENV ${DBNAME}

#=============================================================================#
# BEGIN BACKUP PROCESS #
#=============================================================================#
echo " .........................................................."
echo " STARTING BACKUP ${JOBNAME}"
echo " "
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " Backup Type: ${BACKUP} "
echo " "
echo " \$BACKUPDIROLD: ${BACKUPDIROLD}"
echo " \$BACKUPDIR: ${BACKUPDIR}"

#=============================================================================#
# CHECK DATABASE STATUS #
#=============================================================================#
echo " .........................................................."
echo " CHECKING DATABASE ONLINE STATUS "
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " "

check_database_status

if [ $DBSTATUS = OFFLINE ]; then
# Attempt to startup to build Parametric files
echo " Database is currently OFFLINE "
echo " Attempting to Startup Database to Gather Information..."
$SQL_UTILITY mode=line > /dev/null < connect internal
shutdown immediate;
startup;
exit;
EOF
check_database_status
if [ $DBSTATUS = OFFLINE ]; then
echo " ERROR Starting Database Up...EXITING BACKUP "
echo "${JOBNAME}: Database not online. Exiting backup" >&2
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - **ERROR** Database is not ONLINE" ""
exit
else
echo " Database started up succesfully "
fi
fi
echo " Database is currently ONLINE...continuing with backup"
echo " BACKUP TYPE IS ${BACKUP}"
echo " BACKUP TYPE IS ${BACKUP}"
echo " BACKUP TYPE IS ${BACKUP}"
echo " BACKUP TYPE IS ${BACKUP}"
#================================================#
# SETUP FOR BACKUP #
# #
# This section is executed for both HOT or COLD #
# backups. It is not executed for ARCHS backups.#
# It does the following: #
# #
# 1. Backs up the control file #
# 2. Builds parametric files #
# 3. Moves the alert log file #
# 4. Removes the old backup files #
# 5. Moves the last backup to old backup dir #
# #
#================================================#
#if [ $BACKUP = 'hot' -o $BACKUP = 'cold' ]; then

# Cleanup Parametric Files

cleanup_dbfiles

#
# Backup Control File
#
echo " Backing up controlfile to \$BACKUPDIR/${DBNAME}_control_`date +'%Y"
$SQL_UTILITY mode=line > /dev/null < connect internal
alter database backup controlfile to '${BACKUPDIR}/backup_control_`date +;
exit;
EOF

#
# BUILD PARAMETRIC DATA LISTS
#
# Use SQL to create files containing tablespace, datafile,
# control, archive log and alert log file information. These
# files will be used to generate the necessary commands for the
# backup.
#

echo " Building Parametric Files For Backups"

$SQL_UTILITY mode=line > /dev/null < connect internal
spool $TBLSPACES;
select tablespace_name from sys.dba_tablespaces
order by tablespace_name;
spool off;
spool $DATAFILES;
select file_name from sys.dba_data_files
order by file_name;
spool off;
spool $ALERTLOG;
select value from v\$parameter where name='background_dump_dest';
spool off;
spool $DBFILES;
select file_name from sys.dba_data_files union
select name from v\$controlfile union
select member from v\$logfile union
select p.value||LOWER('/alert_'||d.name)||'.log'
from v\$parameter p, v\$database d
where p.name='background_dump_dest'
order by 1;
spool off;
exit;
EOF

#
# Check the dynamic parameter file size
#

if [ -z $DATAFILES ]; then
echo " ERROR Creating Parametric Files...EXITING BACKUP"
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - ERROR Creating Parametric Files" ""
exit
else
echo " Parametric Files Created Successfully"
fi

#
# MOVE ALERT LOG
#
# The alert log is moved to a new file in the same LOG directory
# before starting another backup.
#

echo " Moving ALERT Log"

while read alertline
do
ALERTDIR=`echo "$alertline" |sed -e 's/---.*$//' -e 's/VALUE.*$//' -e 's/^.`
if [ -n "$ALERTDIR" ]
then
ALERTFILE=${ALERTDIR}/alert_${DBNAME}.log
ALERTNEWFILE=${ALERTDIR}/alert_${DBNAME}.log_`date +'%Y%m%d'`
echo " ALERT LOG DIR: ${ALERTDIR}"
if [ -f ${ALERTNEWFILE} ]
then # FILE Already Exists - Concatenate together
cat ${ALERTFILE} >> ${ALERTNEWFILE}
rm ${ALERTFILE}
else # FILE Doesn't Exist
mv ${ALERTFILE} ${ALERTNEWFILE}
fi
fi
done < ${ALERTLOG}


#
# REMOVE BACKUPDIROLD
#
# Files from the old backup are removed before starting another
# backup job.
#

if [ $BACKUP = 'hot' -o $BACKUP = 'cold' ]; then

echo " .........................................................."
echo " REMOVING OLD BACKUP FILES FROM ${BACKUPOLDDIR}"
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " "

if [ `ls ${BACKUPDIROLD} | wc -l` != 0 ]; then
for I in ${BACKUPDIROLD}/*
do
echo " Removing \$BACKUPDIROLD/`basename ${I}`"
rm $I
done
fi

#
# Move BACKUPDIR to BACKUPDIROLD
# Files from the BACKUP directory are moved to the BACKUP_OLD directory
# before starting another backup job.
#CJH remove OLD Files

echo " .........................................................."
echo " MOVING LAST BACKUP TO ${BACKUPDIROLD}"
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " "

if [ `ls ${BACKUPDIR} | wc -l` != 0 ]; then
for I in ${BACKUPDIR}/*
do
echo " Moving \$BACKUPDIR/`basename ${I}` TO \$BACKUPDIROLD"
rm $I $BACKUPDIROLD
done
fi
fi # End of if [ $BACKUP = 'cold' or 'hot' ]

#======================================================#
# ARCHIVE LOGS BACKUP #
# #
# This section moves the archive logs to the archive #
# log backup directory. #
# #
#======================================================#
if [ "$BACKUP" = archs ]; then
copy_archive_logs "APPEND"
fi

#======================================================#
# COLD BACKUP #
#======================================================#
if [ "$BACKUP" = "cold" ]; then
echo " .........................................................."
echo " SHUTTING DOWN DATABASE "
echo " "
echo " Broadcasting 15 Minute Banner - `date +'[%m/%d/%Y %I:%M:%S%p]'`"

#
# broadcast shutdowns, write your own banners
#
#echo ${BSHUT15} | wall
#sleep 300
#echo " Broadcasting 5 Minute Banner - `date +'[%m/%d/%Y %I:%M:%S%p]'`"
#echo ${BSHUT05} | wall
#sleep 240
#echo " Broadcasting 1 Minute Banner - `date +'[%m/%d/%Y %I:%M:%S%p]'`"
#echo ${BSHUT01} | wall
#sleep 60

touch /home/oracle/${DBNAME}p.maint

#
# Switch the logfile to make sure it gets archived and copied
# and then shutdown the database
#
echo " "
echo " Shutting Down Immediate "

$SQL_UTILITY mode=line > /dev/null < connect internal
alter system switch logfile;
shutdown immediate;
exit;
EOF

#
# Verify the database processes came down
# If it didn't exit that backup process
#
ps -ef | egrep ora | grep "ora_[a-z]*_${DBNAME}"
sleep 5 # Wait for database to shutdown
if [ $? != 0 ]; then
echo " ERROR Shutting Down Database...Exiting "
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - ERROR Shutting Down Database" "${"
exit
fi

#
# Kill sqlnet client connection processes
#
echo " Killing SQLNet V2 Processes "
PROCS=`ps -ef |grep oracle${DBNAME} |grep -v grep |awk '{print $2}'`

if [ x$PROCS = x ]; then
echo " No Processes To Kill $PROCS "
else
echo " Killing Processes $PROCS "
kill $PROCS
fi

#
# Copy Database files to BACKUPDIR
# All files listed in $DBFILES will be copied to the $BACKUPDIR.
#
echo " .........................................................."
echo " COPYING DATABASE FILES TO ${BACKUPDIR} "
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
while read dbfilesline

do
dbfilesline=`echo "$dbfilesline" | sed -e 's/---.*$//' -e 's/FILE_NAME.*$/`
if [ -n "$dbfilesline" ]; then
DBFILENAME=`echo $dbfilesline`
# FSIZE=`ls -lst ${DBFILENAME}|cut -d " " -f15`
FSIZE=`ls -lst ${DBFILENAME} | awk '{print $6}'`
BASEFNAME=`basename ${DBFILENAME}`
echo " "
echo " ........................................ "
echo " [${BASEFNAME} (Size: ${FSIZE}) -Started `date +'%I:%M%p'`]"
echo " Copying ${DBFILENAME} "
echo " Compressing $\{BACKUPDIR}/${BASEFNAME}"
copy_and_compress ${DBFILENAME} ${BACKUPDIR}/${BASEFNAME}
if [ $? != 0 ]; then
echo " ERROR Copying ${DBFILENAME}..Exiting"
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - ERROR Copying ${DBFILENAME}""
exit
fi
fi
done < $DBFILES

#
# Start database back up
#
echo " .........................................................."
echo " STARTING DATABASE UP "
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " "
$SQL_UTILITY mode=line > /dev/null < connect internal
shutdown immediate;
startup;
exit;
EOF

rm -r /home/oracle/${DBNAME}p.maint

fi # End of COLD BACKUP

#================================================#
# HOT BACKUP #
#================================================#
if [ $BACKUP = "hot" ]; then
echo " .........................................................."
echo " COPYING DATAFILES TO ${BACKUPDIR}"
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " \$BACKUPDIR: ${BACKUPDIR} "
echo " "

echo " Broadcasting 1 Minute Banner - `date +'[%m/%d/%Y %I:%M:%S%p]'`"
#echo ${BHOTBAK} | wall
sleep 60

$SQL_UTILITY mode=line > /dev/null << EOF
connect internal
select * from v\$backup;
exit;
EOF

TABSED="sed -e '/selected\.$/d' -e '/^---.*--$/d' -e '/tablespace_name/d' -e "
eval $TABSED | while read TABLESPACE
do
echo " ........................................ "
echo " [TABLESPACE ${TABLESPACE} - Started `date +'%I:%M%p'`]"
$SQL_UTILITY mode=line > /dev/null < connect internal
alter tablespace $TABLESPACE begin backup;
spool $DATAFILES;
select file_name from sys.dba_data_files
where tablespace_name = '$TABLESPACE';
spool off;
exit;
EOF
FILESED="sed -e '/selected\.$/d' -e '/^---.*--$/d' -e '/file_name/d' -e '"

eval $FILESED | while read FILE
do
DATAFILE=`basename $FILE`
FSIZE=`ls -lst ${FILE} | awk '{print $6}'`
# FSIZE=`ls -lst ${FILE}|cut -d " " -f15`
echo " Copying ${FILE} (Size: ${FSIZE})"
echo " Compressing \${BACKUPDIR}/${DATAFILE}"
echo " "
copy_and_compress ${FILE} ${BACKUPDIR}/${DATAFILE}

if [ $? != 0 ]; then
echo "${JOBNAME}: error during file copy $DATAFILE."
$SQL_UTILITY mode=line > /dev/null < connect internal
alter tablespace $TABLESPACE end backup;
exit;
EOF
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - ERROR Copying ${DATAFILE}""
exit
fi

done # End of DataFile Loop
$SQL_UTILITY mode=line > /dev/null < connect internal
alter tablespace $TABLESPACE end backup;
exit;
EOF
done # End of Tablespace Loop

$SQL_UTILITY mode=line > /dev/null < connect internal
select * from v\$backup;
exit;
EOF

copy_archive_logs "REPLACE"
else

FILESED="sed -e '/selected\.$/d' -e '/^---.*--$/d' -e '/^name/d' -e '/^SQLDB

eval $FILSED | while read FILE
do
echo "FILE $FILE"
done

fi # End of HOT BACKUP

#================================================#
# CLEANUP PROCESSES #
#================================================#
# Check Database Status

echo " .........................................................."
echo " CHECKING DATABASE ONLINE STATUS "
echo " Start Time: `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " "

check_database_status
if [ $DBSTATUS = OFFLINE ]; then
echo " ERROR In Database Startup "
echo "${JOBNAME}: error in database startup." >&2
else
echo " Database is Currently Online"
echo " Backing up controlfile to \${BACKUPDIR}/${DBNAME}_control_`date +"
$SQL_UTILITY mode=line > /dev/null < connect internal
alter database backup controlfile to '${BACKUPDIR}/backup_control_`date ;
exit;
EOF
if [ $BACKUP != "archs" ]; then
# echo ${BONLINE} | wall
echo ${BONLINE}
fi
fi # End of Database Online Status

# Cleanup Parametric Files
# cleanup_dbfiles

# Check Disk Space For Next Backup
BAKDIR=`echo $BACKUPDIR|cut -d "/" -f2`
ARCDIR=`echo $BACKUPARC|cut -d "/" -f2`
BAKSPACE=`df -k ${BACKUPDIR}|tail -1|awk '{print $1}'`
ARCSPACE=`df -k ${BACKUPARC}|tail -1|awk '{print $1}'`

(( BAKSPACE = BAKSPACE * 1024 ))
(( ARCSPACE = ARCSPACE * 1024 ))

if [ "$BAKSPACE" -lt "$WARNDISKSIZE" ]; then
echo " WARNING < ${WARNDISKSIZE} Disk Space in /${BAKDIR} For NEXT BACKUP"
echo " WARNING < ${WARNDISKSIZE} Disk Space in /${BAKDIR} For NEXT BACKUP}
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - **WARNING** < ${WARNDISKSIZE} Disk Sp"
fi

if [ "$ARCSPACE" -lt "$WARNDISKSIZE" ]; then
if [ $ARCDIR != $BAKDIR ]; then
echo " WARNING < ${WARNDISKSIZE} Disk Space in /${ARCDIR} For Backup Ar"
echo " WARNING < ${WARNDISKSIZE} Disk Space in /${ARCDIR} For Backup Ar}
$EMAILERROR "${DBNAME}" "[${JOBNAME}] - **WARNING** < ${WARNDISKSIZE} Disk "
fi
fi

echo " .........................................................."
echo " $BACKUP BACKUP COMPLETED - `date +'[%m/%d/%Y %I:%M:%S%p]'`"
echo " .........................................................."
exit

[Robert Salter]

Ali,

I would agree with Patrick and run the script with the "-x", that way you can see where it stalls.

[Ali Imran Abbas]

I dont have much understanding of running the script in debug mode. Can you pls give me some more detail. Also this is a production server, thats why I wanna be on safe side.

[Robert Salter]

You could run the script like this;
sh -x scriptname > /tmp/scriptname.log 2>&1

This should give you a log file that you can check and see where things get stuck.

[TTr]

Which script hangs? The dbbackup.scr or the dbbackup_begin.scr?

You can run the ps command and get the PID of the hung script. Then run the ps command again and grep for the PID of the hung script. This will tell you if and what children processes the hung script has and may give you a clue as to where or why it hangs.