Script required to copy and delete files from source DIR

[Nagu SR]

Hi All,

I have written a script to ensure all my ARCHIVE files are backed up retaining the directory structure.
The Script is--

#!/bin/sh
# Script to take backup of DCS Folders
DCSITIME=$(perl -e '($ss, $mm, $hh, $DD, $MM, $YY) = localtime(time());printf "%04d%02d%02d%02d%02d", $YY + 1900, $MM + 1, $DD, $hh, $mm')
mkdir /backup/destination/backup_files/OTHERS/$DCSITIME

for i in `ls -d /home/ftp/source/APPL/HELLO/*/ARCHIVE|cut -f1,2,3,4,5,6,7 -d /`; do cp -pr $i /backup/destination/backup_files/OTHERS/$DCSITIME/; done

But my requirement is…. I need to take the backup of these files and delete only copied files from original location (Inside ARCHIVE directory there wont be any Sub directories, it will have only files). I cannot delete using normal delete because these files are very dynamic and if I use only `rm ARCHIVE/*`, I have the risk of deleting files which got created between the time of my backup and deletion. I cannot simply move the directory and recreate it because; application may fail at that point of time.

I need help to take the backup with folder structure intact (I’m able to do in the above script) and delete only copied files inside ARCHIVE Folder.


I had gone through

http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1230622095940+28353475&threadId=1114470


But I think that will not help me as files in these DIR are very dynamic and I have chance of loosing one or two files.


How can I do this??


Regards,

Nagu

[Dennis Handly]

>I have chance of losing one or two files.

Any reason you don't change your loop to:
for ...; do
cp -pr $i /backup/destination/backup_files/OTHERS/$DCSITIME/
rm $i
done

[Nagu SR]

Hi,


Here $i is a DIR not file. I dont want DIR to be deleted and recreated.

[VK2COT]

Hello,

This should work.

By the way, you do not need Perl just to
create time-stamped directory (see below).
I added some extra tests because I wwould
hate to delete files if the copy failed :).

The script assumes (as you said) that
there are no subdirectories...

#!/bin/sh
# Script to take backup of DCS Folders

PATH=/usr/bin:/usr/sbin:/sbin:/bin; export PATH

#DCSITIME=$(perl -e '($ss, $mm, $hh, $DD, $MM, $YY) = localtime(time());printf "%04d%02d%02d%02d%02d", $YY + 1900, $MM + 1, $DD, $hh, $mm')
#
# Simpler method, no need for Perl
#
DCSITIME=`date '+%Y%m%d%H%M'`

FROMDIR="/home/ftp/source/APPL/HELLO/*/ARCHIVE"
TODIR="/backup/destination/backup_files/OTHERS/$DCSITIME"

if [ ! -d "$TODIR" ]
then
mkdir -p $TODIR
if [ $? -ne 0 ]
then
echo "ERROR: Cannot create directory $TODIR"
exit 1
fi
fi

for i in `find $FROMDIR -type f`
do
cp -p $i $TODIR
if [ $? -eq 0 ]
then
rm -f $i
else
echo "WARN: Failed to copy $i to $TODIR"
fi
done

exit 0


Cheers,

VK2COT

[Dennis Handly]

>Here $i is a DIR not file. I don't want DIR to be deleted and recreated.

Assuming it is dynamic and only files are created and this script is the only one to remove them:
for ...; do
touch ref_file
cp -pr $i /backup/destination/backup_files/OTHERS/$DCSITIME/
find $i -type f ! -newer ref_file -exec rm -f {} +
done
rm -f ref_file

[Nagu SR]

Hi VK2COT,

With your script, I will not be able to retain DIR structure in target (backup) folder. It just dumps all files in DCSITIME DIR.

But I need to retain my DIR structure in destination
/backup/destination/backup_files/OTHERS/$DCSITIME/*/ARCHIVE/

Can you please suggest?

[VK2COT]

Hello Nagu,

This a bit different request but easy to
resolve anyway:

#!/bin/sh
# Script to take backup of DCS Folders

PATH=/usr/bin:/usr/sbin:/sbin:/bin; export PATH

#DCSITIME=$(perl -e '($ss, $mm, $hh, $DD, $MM, $YY) = localtime(time());printf "%04d%02d%02d%02d%02d", $YY + 1900, $MM + 1, $DD, $hh, $mm')
#
# Simpler method, no need for Perl
#
DCSITIME=`date '+%Y%m%d%H%M'`

FROMDIR="/home/ftp/source/APPL/HELLO/*/ARCHIVE"
TODIR="/backup/destination/backup_files/OTHERS/$DCSITIME"

for i in `find $FROMDIR -type f`
do
DIRNAME=`dirname $i`
NEWTODIR="${TODIR}${DIRNAME}/ARCHIVE"
if [ ! -d "$NEWTODIR" ]
then
mkdir -p $NEWTODIR
if [ $? -ne 0 ]
then
echo "ERROR: Cannot create directory $NEWTODIR"
else
cp -p $i $NEWTODIR
if [ $? -eq 0 ]
then
rm -f $i
else
echo "WARN: Failed to copy $i to $NEWTODIR"
fi
fi
fi
done

exit 0

Cheers,

VK2COT

[VK2COT]

Ahh,

I forgot to add check if the destination directory already exists. Here is the version
with subroutine:

#!/bin/sh
# Script to take backup of DCS Folders

PATH=/usr/bin:/usr/sbin:/sbin:/bin; export PATH

DCSITIME=`date '+%Y%m%d%H%M'`

FROMDIR="/home/ftp/source/APPL/HELLO/*/ARCHIVE"
TODIR="/backup/destination/backup_files/OTHERS/$DCSITIME"

runit () {
cp -p $i $NEWTODIR
if [ $? -eq 0 ]
then
rm -f $i
else
echo "WARN: Failed to copy $i to $NEWTODIR"
fi
}

for i in `find $FROMDIR -type f`
do
DIRNAME=`dirname $i`
NEWTODIR="${TODIR}${DIRNAME}/ARCHIVE"
if [ ! -d "$NEWTODIR" ]
then
mkdir -p $NEWTODIR
if [ $? -ne 0 ]
then
echo "ERROR: Cannot create directory $NEWTODIR"
else
runit $i
fi
else
runit $i
fi
done

exit 0

I am not at work this week, so I could test it on my Fedora 10 server only. It should
be fine on HP-UX too.

VK2COT

[Nagu SR]

Hi,

I could achieve it simply by using cpio command. Thanks for all the help

It can also be achieved by tar as well.

DCSITIME=`date '+%d'`
# Remove old Backup as we need backup for only 30days

rm -rf /backup/test/files/OTHERS/Backup_$DCSITIME
sleep 30
mkdir /backup/test/files/OTHERS/Backup_$DCSITIME

cd /source/Application/
for i in `find */ARCHIVE -type f `;do ls $i|cpio -pvadm /backup/test/files/OTHERS/Backup_$DCSITIME && rm -f $i;done

Regards,

Nagu