1752600 Members
4228 Online
108788 Solutions
New Discussion юеВ

Re: File consistency

 
heaman1
Regular Advisor

File consistency

I have two server in different site , each server have a directory , the files in the directories are as below .

Direcotry in server A
=====================
#ll
-rw-r--r-- 1 root root 68186 Jun 19 15:30 aaa.txt
-rw-r--r-- 1 root root 68186 Jun 19 15:30 bbb.txt
-rw-r--r-- 1 root root 68186 Jun 19 15:30 ccc.txt
-rw-r--r-- 1 root root 68186 Jun 19 15:30 ddd.txt
"

Directory in server B
=====================
#ll
-rw-r--r-- 1 root root 68186 Jun 19 15:30 Ora_aaa.txt
-rw-r--r-- 1 root root 68186 Jun 19 15:30 Ora_bbb.txt
-rw-r--r-- 1 root root 68186 Jun 19 15:30 Ora_ccc.txt
-rw-r--r-- 1 root root 68186 Jun 19 15:30 Ora_ddd.txt
"

Some files will be appended to these directories time by time .

The files size , date and no. of files in these directories should be the same ( except the file name , the files name in server B are begins with Ora_ )

Now , I would like to make sure the files in these two directories are the same and no file is missing , so I would like to regularly check the files is the same or not so that I can take action to correct it immediately .

can you advise the script to check the files in it are the same , eg. check the files regularly ( 4 hrs ) , if any files is missing then notify me so that I can take action to correct it .
16 REPLIES 16
Mel Burslan
Honored Contributor

Re: File consistency

First order of business for you is to setup trust between server A and server B, preferably not as root since everyone can read these files. root trust between servers will get you in trouble with auditors, so stay away from it.

So, let's assume server B trusts server A via ssh keys (you can choose to trust via .rhosts but I advise strongly against it)

OKflag=0
ran=$RANDOM
TMPDIR=/tmp/tempfile.${ran}
mkdir $TMPDIR

cd /my/directory/path
for file in `ls -1`
do
scp server_B:/my/directory/path/Ora_${file} $TMPDIR
r=${?}
if [ $r -ne 0 ]
then
echo "Do something ${file} is missing on server B"
OKflag=1
fi
diff /my/directory/path/${file} $TMPDIR/Ora_${file} > /dev/null
rr=${?}
if [ $rr -ne 0 ]
then
echo "Do some other thing: $file is different on server B"
OKflag=2
fi
rm $TMPDIR/$Ora_{file}
done

if [ $OKflag -eq 0 ]
then
echo "Everything checked out fine at `date`"
fi


Hope this helps

Beware of syntax and typographical errors as the script was not tested in any unix platform.
________________________________
UNIX because I majored in cryptology...
heaman1
Regular Advisor

Re: File consistency

thx Mel Burslan ,

it works fine , for your method , sorry to have two more requirements .


1) as there are so many files in the directoies , it is not possible to copy all files to compare , can advise if only compare the file that created in a certain peroid of time ( eg. 4 hrs / 8 hrs ) what can i do ?

p.s assume the file creation date are the name .


2) can advise if I also want to compare the file size ( to make sure the file consistency ) , is it possible ?


thx

Jitesh purohit_1
Regular Advisor

Re: File consistency

Hi heaman,

You can use scp/rcp to achive the above task

Thanks
Jitesh
Dennis Handly
Acclaimed Contributor

Re: File consistency

>1) as there are so many files in the directories, it is not possible to copy all files to compare, can advise if only compare the file that created in a certain period of time (eg. 4 hrs / 8 hrs) what can I do?

You can create a reference file and only do files that are newer than that file.
If you do "ll -t", you can just stop when you see that file. (Or you can remember the time and do complex date arithmetic to see if newer.)

>2) I also want to compare the file size (to make sure the file consistency)?

From the ll(1) output, you can compare the sizes too.
likid0
Honored Contributor

Re: File consistency

You could also use rsync. it does incremental updates of the files, if their hash has changed.
Windows?, no thanks
heaman1
Regular Advisor

Re: File consistency

" You can create a reference file and only do files that are newer than that file.
If you do "ll -t", you can just stop when you see that file. (Or you can remember the time and do complex date arithmetic to see if newer.) " --->>

sorry , I am not too unstandard what you mean to use "ll -t" to compare the time ? maybe I state more , what I would like is to compare the files that created within 4 hrs because there are too many files in server B , it is not possible to copy , can advise what can i do ? thx
Dennis Handly
Acclaimed Contributor

Re: File consistency

>I am not too understand what you mean to use "ll -t" to compare the time?

Every 4 hours you touch a file. Then you use the previous filename/time to stop looking.
Mel Burslan
Honored Contributor

Re: File consistency

In order to accomplish what you want, I see not other way than both servers trusting each other bidirectionally, i.e., you will need to be able to copy files from one to the other and execute commands via ssh from one to the other. This is necessary because you will have to do a remote command execution to create marker files as Dennis said.

you are on serverB:

touch /var/tmp/4hours_marker

...wait 4 hours or a little less

you are on serverA:

OKflag=0
ran=$RANDOM
TMPDIR=/tmp/tempfile.${ran}
mkdir $TMPDIR

ssh serverB "find /my/directory/path -newer /var/tmp/4hours_marker > /tmp/filelist; touch /var/tmp/4hours_marker"
ssh serverB "scp `cat /tmp/filelist` serverA:/$TMPDIR"

cd /my/directory/path
for file in `ls -1`
do

if [ -a Ora_${file} ]
then
#file exists, compare it to the local file
diff /my/directory/path/${file} $TMPDIR/Ora_${file} > /dev/null
rr=${?}
if [ $rr -ne 0 ]
then
echo "Do something: $file is different on server B"
OKflag=2
fi

else
echo "${file} is missing on serverB. Do something"
OKflag=1
fi

diff /my/directory/path/${file} $TMPDIR/Ora_${file} > /dev/null
rr=${?}
if [ $rr -ne 0 ]
then
echo "Do something: $file is different on server B"
OKflag=2
fi
done

if [ $OKflag -eq 0 ]
then
echo "Everything checked out fine at `date`"
fi

rm -r $TMPDIR

I know there are holes in this procedure, most important of which is lack of extensive error checking but nature of this forums is to guide you down the path. I hope my script helps you wind the way, on which you can build your own script with all the error handling and else.
________________________________
UNIX because I majored in cryptology...
heaman1
Regular Advisor

Re: File consistency

thx Mel,

I tried your method

ssh serverB "find /my/directory/path -newer /var/tmp/4hours_marker > /tmp/filelist; touch /var/tmp/4hours_marker" .

I tried it but in do not create 4hours_marker file , it seems the command have error , if I just want to find the files created in 4 hours , how can i do it ? thx