Operating System - HP-UX
1752805 Members
5366 Online
108789 Solutions
New Discussion юеВ

Re: Shell script dont work with awk.

 
SOLVED
Go to solution
Hein van den Heuvel
Honored Contributor

Re: Shell script dont work with awk.

Dunno what to do with a .rar file. (ok, I do, but I can not right now).
Too specific. zip would be ok, but a plain .TXT file with indication of the inputs, and a example of the desired output MATCHING that input would work best for me, and maybe for others.
Hein.


Ulf Furn
Regular Advisor

Re: Shell script dont work with awk.

Ok, InPut.txt is a plain text file with IP-addresses.
10.45.6.8
192.56.7.9
....
....
....
The file to compare with is a output file from a DB, same deal
10.4.5.7
...
...
...
The script reads those two files into arrays.
And a while loop just reads every line from one file and trys to find a match in the other file. If a match is found it directs the result to the file Identical.txt. That are later used to exclude all identical IP-addresses so I get a file with IP-addresses that should not be in the DB.

Dont know how to explain it more :-(
See attached ZIP file!
Best regards Stefan
Peter Nikitka
Honored Contributor

Re: Shell script dont work with awk.

Hi Stefan,

the term we are trying to explain is the difference between 'match' and 'equal'.
The line
awk 'NR==FNR {ip[$1]=$1} NR>FNR {if(ip[$1]) print "double ip",$1}' seedFile.txt InPut.txt

will NOT do a match, but output equal col1 values.

mfG Peter
The Universe is a pretty big place, it's bigger than anything anyone has ever dreamed of before. So if it's just us, seems like an awful waste of space, right? Jodie Foster in "Contact"
Hein van den Heuvel
Honored Contributor
Solution

Re: Shell script dont work with awk.



The shell script you finally provided is different from what you originally posted.
The awk line has a single '>', and therefor only the output from the final execution will remain.

The match function, will return 0 if there is no match right. And non-0 if it matched ANYWHERE on the line.
You knew that right?

Here is an other example of a fine match which may surprise you:

$ awk 'BEGIN { print match("23.91.181.12","1.1.1.1")}'
5

Anyway, why bother with all this stuff if there is a standard tool ready to do all this??
As I indicated before, just use grep -f

grep -f seedFile.txt InPut.txt > IdenticalIp.txt

grep -v -f seedFile.txt InPut.txt > toBeExcluded.txt

Of course you still have to worry about the . being a wildcard, also for grep.
And you possibly need to worry for the match not being anchored. For example, if someone added a comment :
w.x.y.z # Use to 1.2.3.4

Depending on the grep available you may want use:
-F, --fixed-strings PATTERN is a set of newline-separated strings
-x, --line-regexp force PATTERN to match only whole lines
-w, --word-regexp force PATTERN to match only whole words

Enjoy!
Hein.

Ulf Furn
Regular Advisor

Re: Shell script dont work with awk.

Thank you Hein!
Will go for that soultion.
And no! I did not know that awk match worked like that. Thank you for clarifying that to me :-)

Ulf Furn
Regular Advisor

Re: Shell script dont work with awk.

See above!