FTP Folder cleanup script

gnana prakasam — Mon, 23 Apr 2012 06:52:46 GMT

Hi all,

Pls go through and suggest me in the" #Get the individual recipient from the parsed file..." section

I need the file name like this txt1.txt txt2.txt txt3.txt.

#!/bin/ksh -v
##################################################################################
# Log File manipulation functions
# To be sourced in main job script files
##################################################################################

##################################################################################
# FUNCTION keep_n_most_recent_days
##################################################################################
# Parameters
##################################################################################
# 1.Log File Partial Name (unique application descriptor)
# 2.How many days worth of log file should remain in place
##################################################################################
# Check the application related log files and delete all logs more than n days old
##################################################################################
function keep_n_most_recent_days {

   # Identify log files to be dropped in specified margin and execute deletion...
   find $JOBLOG -name $1\* -atime $2 | xargs rm -f 2>/dev/null

   # Delete any outstanding files out of specificed margin (after the weekend)...
   find $JOBLOG -name $1\* -mtime +$2 | xargs rm -f 2>/dev/null

   return

}
##################################################################################
# FUNCTION keep_n_most_recent_only
##################################################################################
# Parameters
##################################################################################
# 1.Log File Partial Name (unique application descriptor)
# 2.How many most recent log files should remain in place
##################################################################################
# Check the application related log files and delete all but n most recent
##################################################################################
function keep_n_most_recent_only {

   # How many application related log files we have?
   let count=`ls $JOBLOG/$1* | wc -l`

   # If we are to keep more that the existing ones...
   if ((count <= $2)); then

      # ...all existing log files should be kept in that case.
      echo 'not needed'> /dev/null 2>&1

   else

      # How many do we need to delete?
      let files_to_delete=count-$2

      let i=1

      while ((i <= files_to_delete)); do

         # loop through and pick the older log files
         current_file=`ls -tr $JOBLOG/$1* | head -n${files_to_delete} | head -n$i | tail -n1`

         # save the log file names into an array
         command_array[(i-1)]=${current_file}

         let i=i+1

      done

      # Array should have ${#command_array[*]} members

      # Now that we have the old log file name
      # we can go ahead and delete them
      for buffer in ${command_array[*]}
      do
         rm $buffer> /dev/null 2>&1
      done

   fi

   return

}
##################################################################################
# FUNCTION keep_n_most_recent_only_flex
##################################################################################
# Parameters
##################################################################################
# 1.Log File Partial Name (unique application descriptor)
# 2.How many most recent log files should remain in place
# 3.What directory we should look at (improvement over keep_n_most_recent_only)
##################################################################################
# Check the application related log files and delete all but n most recent
##################################################################################
function keep_n_most_recent_only_flex {

   # How many application related log files we have?
   let count=`ls $3/$1* | wc -l`

   # If we are to keep more that the existing ones...
   if ((count <= $2)); then

      # ...all existing log files should be kept in that case.
      echo 'not needed'> /dev/null 2>&1

   else

      # How many do we need to delete?
      let files_to_delete=count-$2

      let i=1

      while ((i <= files_to_delete)); do

         # loop through and pick the older log files
         current_file=`ls -tr $3/$1* | head -n${files_to_delete} | head -n$i | tail -n1`

         # save the log file names into an array
         command_array[(i-1)]=${current_file}

         let i=i+1

      done

      # Array should have ${#command_array[*]} members

      # Now that we have the old log file name
      # we can go ahead and delete them
      for buffer in ${command_array[*]}
      do
         rm $buffer> /dev/null 2>&1
      done

   fi

   return

}
##################################################################################
##################################################################################
# FUNCTION TO DELETE FILES IN FTPOUT FOLDER
##################################################################################
# Parameters
##################################################################################
# 1.FTP File Partial Name (unique application descriptor)
# 2.How many days worth of FTP file should remain in place
# 3.FTP Server Name
# 4.FTP User Name
# 5.FTP Password
# 6.FTP LOG File
##################################################################################
# Check the application related FTP files and delete all FTP more than n days old
##################################################################################
function keep_FTP_n_most_recent_days {

   # Identify ftp files to be dropped in specified margin and execute deletion...
   find $FTPOUT -name $1\* -atime $2 |awk -F/ '{print $NF}' > $FTPOUT/FTP_Delete.txt
   find $FTPOUT -name $1\* -atime $2 | xargs rm -f 2>/dev/null

   # Delete any outstanding files out of specificed margin (after the weekend)...
   find $FTPOUT -name $1\* -mtime +$2 |awk -F/ '{print $NF}' >> $FTPOUT/FTP_Delete.txt
   find $FTPOUT -name $1\* -mtime +$2 | xargs rm -f 2>/dev/null

   #Get the line number, parse out empty spaces and empty lines...

   let count=`cat $FTPOUT/FTP_Delete.txt | tr -s "\012" | tr -d " " | wc -l`

   #Reset counter
   let i=1

   #Loop in...
   while ((i <= count)); do

        #Get the individual recipient from the parsed file...
        filename=`cat $FTPOUT/FTP_Delete.txt | tr -s "\012" | tr -d " " | head -n$i | tail -n1`
        #Time to call the delete ftp file function...
        delete_FTP_Files $3 $4 $5 $filename $6

        let i=i+1

   done

   return

}
#################################################################################
##################################################################################
##################################################################################
# FUNCTION TO DELETE FILES IN FTP FOLDER
##################################################################################
# Parameters
##################################################################################
# 1.FTP Server Name
# 2.FTP User Name
# 3.FTP Password
# 4.FTP File Name
# 5.FTP LOG File
##################################################################################
# Delete the FTP files from the destination
##################################################################################
function delete_FTP_Files {

FTPServer=$1
FTPUserID=$2
FTPUserPswd=$3

echo $4
echo $5
#FTP Command Section
ftp -v -n<<EOF>$5
   open $FTPServer
   user $FTPUserID $FTPUserPswd
   delete $4
bye
EOF

}
#################################################################################

Re: FTP Folder cleanup script

Dennis Handly — Mon, 23 Apr 2012 08:08:08 GMT

>I need the file name like this txt1.txt txt2.txt txt3.txt.

What do you mean? All on one line? Or only with the string "txt"?

Some optimizations you can make:

>find $JOBLOG -name $1\* -atime $2 | xargs rm -f 2>/dev/null
>find $JOBLOG -name $1\* -mtime +$2 | xargs rm -f 2>/dev/null

Combine the two:

find $JOBLOG -name $1\* $ -atime $2 -o -mtime +$2 $ -exec rm -f {} + 2>/dev/null

Instead of using let, just use (( )):

(( files_to_delete=count-$2 ))

Instead of using this pattern, use while read:

> while ((i <= files_to_delete)); do
> current_file=`ls -tr $JOBLOG/$1* | head -n${files_to_delete} | head -n$i | tail -n1`

ls -tr $JOBLOG/$1* | head -n${files_to_delete} | while read file; do

I don't think you need a for-loop here:

>for buffer in ${command_array[*]}

rm -f "${command_array[@]}" 2> /dev/null

You probably want to combine these so you don't search 4 times:

   find $FTPOUT -name $1\* -atime $2 |awk -F/ '{print $NF}' > $FTPOUT/FTP_Delete.txt
   find $FTPOUT -name $1\* -atime $2 | xargs rm -f 2>/dev/null
   find $FTPOUT -name $1\* -mtime +$2 |awk -F/ '{print $NF}' >> $FTPOUT/FTP_Delete.txt
   find $FTPOUT -name $1\* -mtime +$2 | xargs rm -f 2>/dev/null

I.e. Do one find and put it into a file. Use that with xargs to remove. Then pass through awk.

topic Re: FTP Folder cleanup script in Operating System - HP-UX

FTP Folder cleanup script

Re: FTP Folder cleanup script