topic Check file script in Operating System - HP-UX

Check file script

ust3 — Tue, 04 Dec 2007 07:11:20 GMT

# vi file_for_compare
file1.txt;200712031200
file2.txt;200712041457
file3.txt;200712041451
file4.txt;200712051512

I have a file as above , the format is file name + date & time , I would like to compare it with the files in a directory as below

#ls /tmp/directory_for_compare
-rw-r--r-- 1 user edp 4324 Dec 04 14:57 file1
-rw-r--r-- 1 user edp 4324 Dec 04 14:57 file3
-rw-r--r-- 1 user edp 4324 Dec 05 18:57 file12

I would like to find out which file is missing for everyday ( what I want to find out is file name that the file_for_compare have but directory_for_compare don't have and vice versa ) . For example , assume today is 04-Dec , so only compare the file which the creation date is 04-Dec , in the file_for_compare , there are file2 and file3 need to compare , in the directory_for_compare , there are file1 and file3 need to compare ( because only these files creation date is 04-Dec ) , so I would like the output like below :

file_for_compare have but directory_for_compare do not have
"file2"

file_for_compare do not have but directory_for_compare have
"file1"

If today is 5-Dec , then only compare 5-Dec, if today is 6-Dec then only 6-Dec etc , no date input is required .

can advise how to write the script ? thx in advance.

Re: Check file script

Dennis Handly — Tue, 04 Dec 2007 07:41:00 GMT

You need to convert the date to two separate search patterns. TODAY1 would have problems for files over 6 months old.

TMP=/var/tmp/dc.$$

TODAY1=$(date +"%b %d")
TODAY2=$(date +"%Y%m%d")

echo $TODAY1 # debugging
echo $TODAY2 # debugging

awk -F";" -v PAT=$TODAY2 '
$2 ~ PAT { print $1 }' file_for_compare | sort > $TMP.fff

ll /tmp/directory_for_compare | awk -v PAT="$TODAY1" '
$0 ~ PAT { print $NF }' | sort > $TMP.dff

echo "Files not in directory:"
comm -23 $TMP.fff $TMP.dff
echo "Files not in control file:"
comm -13 $TMP.fff $TMP.dff

rm -f $TMP.fff $TMP.dff

Re: Check file script

ust3 — Tue, 11 Dec 2007 04:00:05 GMT

thx reply,

I tried the above script , but there is no output , I would like to ask what is PAT mean ? do I still need to modify before use this script ?

thx

Re: Check file script

Dennis Handly — Tue, 11 Dec 2007 04:37:04 GMT

>I tried the above script but there is no output

The script won't give output if "today" isn't in Dec 3 though Dec 5. You said:
If today is 5-Dec, then only compare 5-Dec, if today is 6-Dec then only 6-Dec etc, no date input is required.

I only check the files for today, not every file and entry in file_for_compare. Also because of that, I don't bother checking hours and minutes. (Running the script at midnight would have problems.)-:

>would like to ask what is PAT mean? do I still need to modify before use this script?

PAT is the awk string variable that signifies "today" by an exact match. There are two date "styles" for the two data sources.

Re: Check file script

ust3 — Tue, 11 Dec 2007 05:37:43 GMT

thx reply ,

I found there is single ' sign in the below statement , is it correct ? thx

awk -F";" -v PAT=$TODAY2 '
$2 ~ PAT { print $1 }' file_for_compare | sort > $TMP.fff

Re: Check file script

Dennis Handly — Tue, 11 Dec 2007 05:52:27 GMT

>I found there is single ' sign in the below statement, is it correct?
awk -F";" -v PAT=$TODAY2 '
$2 ~ PAT { print $1 }' file_for_compare | sort > $TMP.fff

Yes, one on the first line to start the awk script and the other on the last line to end it. I usually break up complex awk scripts like that but since it is only one line, you can put them together.

Re: Check file script

ust3 — Fri, 14 Dec 2007 15:13:18 GMT

thx reply ,

I would like to modify something , if I NOT ONLY compare today's file , I would like to compare ALL file name , no matter what date it is , as below file , there are different date ( 03Dec , 04Dec & 05Dec ) , I would like to compare all these files , can advise how to do it ? thx

file1.txt;200712031200
file2.txt;200712041457
file3.txt;200712041451
file4.txt;200712051512

ps. the file name is unique .

Re: Check file script

Dennis Handly — Sat, 15 Dec 2007 06:52:51 GMT

>I would like to compare ALL file name

(If you want to ignore the date parts completely, ignore lots of this.)

Well first you have to convert from one format until the other so you can just use comm(1) to compare the files. I would suggest you convert file1.txt;200712031200 to:
file1.txt Dec 03 12:00
(This won't work for files older than 6 months, without more work. It will also fail if you aren't in the American Nerd locale (C).)

Let us assume you have file_for_compare and the output of that "ll /tmp/directory_for_compare" in a file, directory_for_compare.

Should I assume that file1.txt is the same as file1?
Did you want to compare by just the name or including the date?
TMP=/var/tmp/dc.$$
awk -F";" '
BEGIN {
mon[1] = "Jan"; mon[2] = "Feb"; mon[3] = "Mar"; mon[4] = "Apr"
mon[5] = "May"; mon[6] = "Jun"; mon[7] = "Jul"; mon[8] = "Aug"
mon[9] = "Sep"; mon[10] = "Oct"; mon[11] = "Nov"; mon[12] = "Dec"
}
{
name = $1
i = index(name, ".txt")
if (i != 0)
name = substr(name, 1, i - 1)
year = substr($2, 1, 4)
Mon = substr($2, 5, 2)
day = substr($2, 7, 2)
#hr = substr($2, 9, 2)
#min = substr($2, 11, 2)
print name, mon[Mon], day #, hr ":" min
} ' file_for_compare | sort > $TMP.fff

#-rw-r--r-- 1 user edp 4324 Dec 04 14:57 file1
awk '
{
print $9, $6, $7 #, $8
} ' directory_for_compare | sort > $TMP.dff

echo "Files not in directory:"
comm -23 $TMP.fff $TMP.dff
echo "Files not in control file:"
comm -13 $TMP.fff $TMP.dff

rm -f $TMP.ffc $TMP.dfc

Re: Check file script

ust3 — Mon, 17 Dec 2007 11:23:08 GMT

thx reply,

sorry , I have a last requirement .

I sure that the date in the control file MUST be the same date , for example , like the below file , all files are the SAME date (3-Dec) . But it may be not today's date (today is 17-Dec , but the date in the control file is 3-Dec) .

file1.txt;200712031200
file2.txt;200712031457
file3.txt;200712031451
file4.txt;200712031512
" ;200712031551
" ;200712031541
" ;200712031554

so I would like to compare the date ( in the above example , it is 3-Dec ) in the control file with the files that are the same date ( so it is also 3-Dec) in the directory .

ps. the new script can let me not only comparing today's file , but also let me to do the comparsion everyday after I received the control file .

Thx again.

Re: Check file script

Dennis Handly — Tue, 18 Dec 2007 00:07:42 GMT

>I have a last requirement. Insure that the date in the control file MUST be the same date

My last script does that. If you want to compare the time you need to remove the "#".
#hr = substr($2, 9, 2)
#min = substr($2, 11, 2)
print name, mon[Mon], day #, hr ":" min

Re: Check file script

ust3 — Tue, 18 Dec 2007 01:59:27 GMT

thx reply ,

but when I run the script , it pop the error,

awk: cmd. line:5: fatal: file `directory_for_compare' is a directory .

can advise what is wrong ? thx

Re: Check file script

Dennis Handly — Tue, 18 Dec 2007 02:46:32 GMT

>but when I run the script, it pop the error,
awk: cmd. line:5: fatal: file `directory_for_compare' is a directory .

That was a file that contained the ll(1) output. Just change those lines to:
ll directory_for_compare | awk '
{
print $9, $6, $7 #, $8
} ' | sort > $TMP.dff

Re: Check file script

ust3 — Tue, 18 Dec 2007 04:14:19 GMT

Hi Dennis,

thx your reply and reply again.

I try the your scirpt , we have two problem.

1. As my request above all files date in control file is 3-Dec , so I only want to check the files with the SAME date (3-Dec also ) in the directory ( because there are many different date files in the directory , I don't want to list all of them ) ; on the same way , if the file date is 4-Dec in control file , so it compare the files that are 4-Dec in the directory ;
2. there are many file with various type in the diectroy , if I only want to compre .txt , can advise what can i do ?

thx

Re: Check file script

Dennis Handly — Tue, 18 Dec 2007 05:45:27 GMT

>1. I only want to check the files with the SAME date (3-Dec also) in the directory,

That's what the script does. Compares both the name and the date. And then does this for ALL entries in the control file and directory.

Otherwise I need a specific example of what you want.

>2. if I only want to compare .txt, can advise what can i do?

It would help if you listed a new control file and directory. But if you only want to compare .txt files, you use "ll *.txt".

Re: Check file script

ust3 — Thu, 20 Dec 2007 02:53:09 GMT

thx Dennis and sorry to ask again,

may be I not clearly state my requirement , your script works fine for using the control file to compare the files in directoty , like the above example , the date is 3-Dec , it can list out the files that are in the control file but not in directory , it works fine . Now , my requiremnet is that except this function , I also want to compare the same date (so it is 3-Dec) that the files in the directory but it is not exist in the control file .

the below is the part of the script , it list ALL files in the directory but not only 3-Dec , can advise how to change it ? thx

ll directory_for_compare | awk '
{
print $9, $6, $7 #, $8
} ' | sort > $TMP.dff

ps. why use 3-Dec , it is because the date in the control file is 3-Dec.

Re: Check file script

ust3 — Thu, 20 Dec 2007 03:05:25 GMT

hi Dennis,

I think the first script provided is perfect , what need to change is TODAY1 and TODAY2 is not today's date , it should be 3-Dec , can advise how to change it ? thx

TMP=/var/tmp/dc.$$

TODAY1=$(date +"%b %d")
TODAY2=$(date +"%Y%m%d")

echo $TODAY1 # debugging
echo $TODAY2 # debugging

awk -F";" -v PAT=$TODAY2 '
$2 ~ PAT { print $1 }' file_for_compare | sort > $TMP.fff

ll /tmp/directory_for_compare | awk -v PAT="$TODAY1" '
$0 ~ PAT { print $NF }' | sort > $TMP.dff

echo "Files not in directory:"
comm -23 $TMP.fff $TMP.dff
echo "Files not in control file:"
comm -13 $TMP.fff $TMP.dff

rm -f $TMP.fff $TMP.dff

Re: Check file script

Dennis Handly — Thu, 20 Dec 2007 05:51:53 GMT

>I want to compare the same date (so it is 3-Dec) that the files in the directory but it is not exist in the control file.

So instead of picking "today" or ALL dates, you want to give a specific date?

If so, that would have to be a parm to the script.

>it list ALL files in the directory but not only 3-Dec, can advise how to change it?

You would need to go back to the first script.

>I think the first script provided is perfect, what need to change is TODAY1 and TODAY2 is not today's date, it should be 3-Dec, can advise how to change it?

Well, you can work hard and provide both formats to your script:
$ script 20071203 "Dec 03"

Then the script would have:
TODAY1=$1
TODAY2=$2

Or you could take the YYYYMMDD format and generate a Dec 03 format:
TODAY2=$1 # Script arg
#YYYY=${TODAY2%????}
DD=${TODAY2#??????}
MMDD=${TODAY2#????}
MM=${MMDD%??}
set -A MON Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
TODAY1="${MON[MM-1]} $DD"
echo $TODAY1
echo $TODAY2

Re: Check file script

ust3 — Mon, 07 Jan 2008 08:54:51 GMT

thx dennis ,

back to your first script

====================================================================
TMP=/var/tmp/dc.$$

TODAY1=$(date +"%b %d")
TODAY2=$(date +"%Y%m%d")

echo $TODAY1 # debugging
echo $TODAY2 # debugging

awk -F";" -v PAT=$TODAY2 '
$2 ~ PAT { print $1 }' file_for_compare | sort > $TMP.fff

ll /tmp/directory_for_compare | awk -v PAT="$TODAY1" '
$0 ~ PAT { print $NF }' | sort > $TMP.dff

echo "Files not in directory:"
comm -23 $TMP.fff $TMP.dff
echo "Files not in control file:"
comm -13 $TMP.fff $TMP.dff

rm -f $TMP.fff $TMP.dff

========================================================================

it works fine when you post it , I have tried it and get the perfect result , but it is strange that I can't get the result from the part below,

ll /tmp/directory_for_compare | awk -v PAT="$TODAY1" '
$0 ~ PAT { print $NF }' | sort > $TMP.dff

nothing is generated to $TMP.dff , is it because the year is changed to 2008 now ? thx for your advise.

Re: Check file script

Dennis Handly — Mon, 07 Jan 2008 19:49:01 GMT

>it works fine when you post it

It only works for files with today's date. You have to look at the other script versions if you want other dates.

>I can't get the result from the part below,
ll /tmp/directory_for_compare | awk -v PAT="$TODAY1" '...
>nothing is generated to $TMP.dff, is it because the year is changed to 2008 now?

You need to show some example filenames, (ll of that directory). 2008 shouldn't matter unless the file is over 6 months old.

It would help if you can clearly state what you now want to do and provide some example data.

Re: Check file script

ust3 — Tue, 08 Jan 2008 02:39:36 GMT

thx Dennis ,

You need to show some example filenames, (ll of that directory). 2008 shouldn't matter unless the file is over 6 months old.
It would help if you can clearly state what you now want to do and provide some example data.

What I want now is that the requirement of my first question ( please ignore my replies except the first post) , I have tried your script when you post it , it works fine , but now when I run it , the .dff is empty , I sure in the directory "directory_for_compare" , there is file which is today's date ,
I found that the output of date +"%b %d" is --> Jan 08
the output of ll directory_for_compare is --> Jan 8 , there is difference in output , is it the reason of the problem ? thx

-rw-r--r-- 1 user edp 4324 Dec 4 14:57 file1
-rw-r--r-- 1 user edp 4324 Dec 4 14:57 file3
-rw-r--r-- 1 user edp 4324 Dec 5 18:57 file12
-rw-r--r-- 1 user edp 4324 Dec 8 07:42 file12