help in removing entries from a file

[lawrenzo_1]

Guys,

I could do with some help:

I have a file with a list of entries that I want to remove from a logfile:

iprange1
iprange2
iprange3
iprange4

what would be the best way to remove these from a logfile with many ipranges in?

Thanks

Chris

[Oviwan]

Hey

something like that:

$cat logfile | grep -v ipranges > temp.file
$cat temp.file > logfile

or what syntax have your iprange?

Regards

[whiteknight]

Hi Lawrenzo,

See also
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1156808


WK

[lawrenzo_1]

ok yes I know these options however what I am looking for is instead of doing:

grep -v iprange1 |grep -v iprange2 |grep -v iprange3 etc etc

is have a list of each different iprange that I do not what to display from the logfile in a file then use this to not display from the log:

ie

how do I have a file with

iprange1
iprange2
iprange3
iprange4


then not display this information from the logfile that has multiple entries of

iprange1
iprange2
iprange3
iprange4
iprange5
iprange6
iprange7
iprange8

basically the ipranges are exeptions from a report that is emailed daily.

Thanks chaps.

[James R. Ferguson]

Hi Chris:

# grep -v -f pattern_file data_file

...where 'pattern_file' contains the strings that you do *not* want to include in your output.

Please consider also evaluating your previous thread on this subject:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1156808

Regards!

...JRF...

[James R. Ferguson]

Hi (again) Chris:

Actually for *ranges* you have 'grep's ability to use extended regular expressions ('-E') in your pattern file ('-f)' [see the manpges for 'grep']:

For example:

# cat .pattern_file
10.11.*.2

# cat .data_file
10.11.12.1
10.11.12.2
10.11.12.3
10.11.13.1
10.11.13.2
10.11.13.3
10.11.14.1
10.11.14.2
10.11.14.3

# grep -v -E -f .pattern_file data_file
10.11.13.1
10.11.13.3
10.11.14.1
10.11.14.3

Regards!

...JRF...

[James R. Ferguson · Accepted solution]

Hi Chris:

DISREGARD MY LAST POST! ENOCOFFEE! We need to escape the 'dot' if we want it to mean itself and not any character in the context I presented.

Actually for *ranges* you have 'grep's ability to use extended regular expressions ('-E') in your pattern file ('-f)' [see the manpges for 'grep']:

For example:

# cat .pattern_file
10\.11\.[0-9]*\.2

# cat .data_file
10.11.12.1
10.11.12.2
10.11.12.3
10.11.13.1
10.11.13.2
10.11.13.3
10.11.14.1
10.11.14.2
10.11.14.3

# grep -v -E -f ./pattern_file ./data_file
10.11.12.1
10.11.12.3
10.11.13.1
10.11.13.3
10.11.14.1
10.11.14.3

# grep -E -f ./pattern_file ./data_file
10.11.12.2
10.11.13.2
10.11.14.2

Regards!

...JRF...

[Peter Nikitka]

Hi,

just as JRF remarked: make yourself clear, if you want to consider 'iprange1' as string or as pattern.
Note, that using it as pattern creates the problem, that you have to watch for wildcard characters: something like
1.2.3.4
will match
152.3.4
as well, because '.' is 'any char' when using it in regular expressions! Best use
1[.]2[.]3[.]4
for such a case.
Important as well is the fact, if your pattern are anchored at begin or end of a line. Taking this into account can speed up your script dramatically!

Second: NOT wanting a pattern match, use
fgrep (or grep -F) - this does pure string compares.

mfG Peter

[lawrenzo_1]

Thanks all,

I am not too familiar with fgrep or egrep however do understand these are powerful tools and will check and test out.

Chris.

[Dennis Handly]

>JRF: extended regular expressions

No need to use the -E hammer, your regular expressions (*) aren't extended.

>Peter: Best use 1[.]2[.]3[.]4

Usually they are quoted with a "\".

>if your pattern are anchored at begin or end of a line. Taking this into account can speed up your script dramatically!

Right. You can do that with grep -x but the logfile could only have that number and no other data on each line.

I'm surprised Peter didn't mention the gotcha with fgrep, 1.2.3.4 will match 21.2.3.4

So you need to use grep -w, or use anchored strings in the file. Given't JRF's file:
10\.11\.[0-9]*\.2

You could use vi to add the anchors:
:%s/^/^/
:%s/$/$/