script to match character

[Intothenewworld]

I have a master file (master.txt), the content is as below

#vi master.txt
aaa
bbb
ccc
ddd
eee
fff
ggg

I also have some file (eg. file1.txt , file2.txt , file3.txt ... ) , these files may be have the characters as above , for example
#vi file1.txt
123456 aaa 54

#vi file2.txt
1456456 bbb 452

#vi file3.txt
667766 ccc 123

like above , file1.txt have aaa , file2.txt have bbb , file3.txt have ccc , I would like to sort it out , to output a file (matching.txt) that match the file name with the character as below .

#vi matching.txt
aaa file1.txt
bbb file2.txt 
ccc file3.txt
ddd "
eee "
fff "
ggg "

can advise what can i write the script ?

Thanks in advance.

[Intothenewworld]

thx reply ,

may be I simpifiy my requirement , if I just would like to list out the line ( in the master file  ) that do not appear in any files , eg. if aaa is not exist in any files , then output aaa ; if bbb is exist in file1.txt (any files would be ok , assume exist in file1.txt now ) , then do not output bbb .

can advise what can i do ?

thx

[Dennis Handly]

>if I just would like to list out the line that do not appear in any files, eg. if aaa is not exist in any files, then output aaa

You could do something like:

while read line; do

   fgrep -q "$line" $filelist

   if [ $? -ne 0 ]; then

      echo "$line does not appear"

   fi

done < master.txt

Where $filelist contains the names of the files you want to search.

[Intothenewworld]

thx reply,

Where $filelist contains the names of the files you want to search. <== how to set this variable ?

I just would like to output a file to show all lines that do NOT appear in any file .

Thanks. 

[Intothenewworld]

may be I explain my requirement in more detail , if I just would like to list out the line ( in the master file  ) that do not appear in any files , eg. if aaa is not exist in any files , then output aaa to the output file ; if bbb is exist in file1.txt (any files would be ok , assume exist in file1.txt now ) , then do not output bbb to the output file ; if ccc is not exist in any files , then output ccc to the output file ..

I just would like to output a file to show all lines that do NOT appear in any file .

the output file as below
#vi output.txt
aaa
ccc

can advise what can i do ?

[Dennis Handly]

>Where $filelist contains the names of the files you want to search. <== how to set this variable?

If you know the names, you can just list them:

filelist="file1.txt file2.txt file3.txt"

If you just want to hard code them, you can also do that.  Or you can find patterns:

filelist="file*.txt"

>if I just would like to list out the line (in the master file) that do not appear in any files

This should do it:

while read line; do

   fgrep -q "$line" file*.txt

   if [ $? -ne 0 ]; then

      echo "$line"

   fi

done < master.txt

[Intothenewworld]

thanks for Dennis Handly , you help a lot.

May be I was not express my requirement clearyly and I would like to change a bit requirement .

I have a file as below ( eg. master.txt) , it delimited by " and space

#vi master.txt
"""""""""  aaa bbb ccc ddd eee " " "
"""""""""  xxx yyy zzz mmm ooo " " "
"""""""""  ggg hhh iii jjj kkk " " "

I also have a set of files (eg. file1.txt , file2.txt ... ) .

what I would like to do is to list the lines that do not appear the fourth column ( ddd , mmm , jjj ) of master.txt in any files, that mean search ddd , mmm , jjj in all .txt file ,if do not appear , then output the line.

Assume  mmm , jjj is not appear in any .txt file , then output the below result.

"""""""""  xxx yyy zzz mmm ooo " " "
"""""""""  ggg hhh iii jjj kkk " " "

can advise how I can do for it ?

[Dennis Handly]

>May be I did not express my requirement clearly

Did you try it?

>I have a file as below, it delimited by " and space

"""""""""  aaa bbb ccc ddd eee " " "

Can you explain how the double quote is delimiting things?

As far as the shell's read is concerned (with the default IFS), there are these 9 fields:

""""""""", aaa, bbb, ccc, ddd, eee, ", ", "

>I would like to do is to list the lines that do not appear the fourth column (ddd, mmm, jjj) of master.txt in any files

Try:

while read f1 f2 f3 f4 f5 f6 f7 f8 f9; do

   fgrep -q "$f5" file*.txt

   if [ $? -ne 0 ]; then

      echo "$f1 $f2 $f3 $f4 $f5 $f6 $f7 $f8 $f9" # print an approximation of original line

   fi

done < master.txt

[Intothenewworld]

Thanks , it should be works

May I ask
1) what is the use of f1 , f2 .... ? is it represent " ?
2) what is the mean of "$f5

[Dennis Handly]

>Thanks, it should be works

If you are happy with the answers, please click on the Kudos stars of each answer that was helpful.

>1) what is the use of f1, f2 ...? is it represent?

Since I don't know what your fields means, I just assigned short variable names, they just mean field1, etc.

>2) what is the meaning of "$f5?

This provides the value of the variable f5, field5.

[Intothenewworld]

thx replies,

I am sorry that when I apply it to my real case , it still have problem , as in my real case , the master file has a bit difference Dennis Handly 's method is delimited by space only ( f1 f2 f3 is delimited by space ) , the real file is a bit complicate.

The below is the master file of the real case , in the string (eg. aaa bbb ccc ddd eee) , there are other space before this string .

#vi master.txt
"ffd" hg " gg""""""  aaa bbb ccc ddd eee " " "
"fd "jg "" """""  xxx yyy zzz mmm ooo " " "
"hdgd"jg"""fd""f""  ggg hhh iii jjj kkk " " "

I want to compare ( in my master file ) is

1) after 9 " , then
2) the 4th column of the string that delimited by space ,

can advise what can i do ?

thx

[Intothenewworld]

I may be make it confued so I would like to clearly re-post my question .

I have a master file , the content is as below.
#vi master.txt
"d"""""""" aaa bbb ccc ddd eee " " "
"""y""d"""" xxx yyy zzz mmm ooo " " "
""f""""""" ggg hhh iii jjj kkk " " "

I also have some text files ,
eg.
#vi file1.txt
aaa
#vi file2.txt
bbb
#vi file3.txt
ccc
#vi file4.txt
ddd

I would like to check if the fourth column ( eg. ddd mmm jjj ) is exist in these text files , if not exist , then output this line , in this case , output the below ( as ddd exists in file4.txt )
"""y""d"""" xxx yyy zzz mmm ooo " " "
""f""""""" ggg hhh iii jjj kkk " " "

The requirement is check
1) after 9 " , and
2) the 4th column of the string that delimited by space .

can advise what can I do ? Thanks

[Dennis Handly]

>1) after 9 ", then
>2) the 4th column of the string that delimited by space

Ok, try this:

while read line; do
   IFS_save=$IFS
   IFS='"\n'    # split on double quote
   echo "$line" | read f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11
   IFS=$IFS_save
   f10=${f10# *}  # strip leading spaces
   f10=${f10% *}  # strip trailing spaces
   echo "$f10" | read f1 f2 f3 f4 f5  # split on spaces
   fgrep -q "$f4" file*.txt
   if [ $? -ne 0 ]; then
      echo "$line" # print original line if field not found
   fi
done < master.txt