Languages and Scripting

Searching Errors in a log file

 
SOLVED
Go to solution
Shivkumar
Super Advisor

Searching Errors in a log file

Hi,

I am looking out for a shell script to accomplish the below :-

We want to capture each occurance of the word Error ( case insensitive ) in a log file and 5 lines below each of the lines and store the output in a file.

Can anyone post the script ?

Thanks,
Shiv
19 REPLIES 19
Peter Nikitka
Honored Contributor
Solution

Re: Searching Errors in a log file

Hi,

Some open issues:
- you say 'word': is it really a separate word (white space around)? 'Error:' for example would NOT meet this criterium.
- you say case insensitive: may the be ANY combinations of Error, ErrOR or is there a restriction to Error, error and ERROR?

I offer an awk solution (untested).

awk '/Error/ || /error/ || /ERROR/ {count=6}
count {print;count--}' infile >outfile

mfG Peter
The Universe is a pretty big place, it's bigger than anything anyone has ever dreamed of before. So if it's just us, seems like an awful waste of space, right? Jodie Foster in "Contact"
john korterman
Honored Contributor

Re: Searching Errors in a log file

Hi,

try the attached script with these parameters: $1="Error" , $2="5", $3="path_to_file"

regards,
John K.
it would be nice if you always got a second chance
Hein van den Heuvel
Honored Contributor

Re: Searching Errors in a log file


Perl regular expression have something a 'break' match between words: \b

It helps to make sure that a script like requested does trigger on "Error:" but not on "terrorist".

Example usage:

perl -ne "$count=5 if /\berror\b/i; print if $count-- > 0" your-log-file

hth,
Hein.
Bill Hassell
Honored Contributor

Re: Searching Errors in a log file

And just a note about other 'error' conditions. You might want to search for other keywords that also signify problems:

grep -i -e fail -e warn -e alert -e critical -e abort /some-dir/some-log

The above grep just shows the line where one or more of the search strings match. Use it to see if there are any other messages that might be important but do not have Error in the entry.


Bill Hassell, sysadmin
James R. Ferguson
Acclaimed Contributor

Re: Searching Errors in a log file

Hi Shiv:

Bill makes a good point, and we can easily use Hein's Perl snippet thusly:

# perl -ne '$count=5 if /\b(error|warn|critical|alert|abort|fail)\b/i;print if $count-- > 0' logfile

Thus if the file (or files) you specify as arguments contain any of the words "error", "warn", "critical", "alert", "abort" or "fail", the line on which any one occurs will be printed and four more lines afterwards.

As written, you can process multiple files at one simply by passing their names as arguments. If you would like to do this, *and* keep track of the filename for which matches occur, modify the script thusly:

# perl -ne '$count=5 if /\b(error|warn|critical|alert|abort|fail)\b/i;print "$ARGV:$_" if $count-- > 0' log1 log2 log3

...Now output can look like:

log1:critical alert: line-3
log1:line-4
log1:line-5
log1:line-6
log1:line-7
log2:ERROR at line-6
log2:line-7
log2:line-8
log2:line-9
log2:line-10

Regards!

..JRF...
Rory R Hammond
Trusted Contributor

Re: Searching Errors in a log file

This AWK example works hpux 11.11
I put a space before each word.
so it would get things like " Error:"

awk -F: '
(tolower($0) !~ / error| warn| alert| critical| abort| invalid/) {next}
{
print "\n" $0
for (i=1;i < 6;i++)
{getline
print $0
}
} ' /var/adm/syslog/syslog.log
There are a 100 ways to do things and 97 of them are right
James R. Ferguson
Acclaimed Contributor

Re: Searching Errors in a log file

Hi Rory:

Yes, placing a space (blank) character before the words in the 'awk' solution helps, but it dones *not* solve the number of matches that Perl does.

For starters, what if the space were a tab character? The use of '\b' in Perl regular expressions covers word boundries delimited by spaces, tabs and most punctuation characters like ":" and "-" that might appear like "alert-bad thing happened".

Regards!

...JRF...
Coolmar
Esteemed Contributor

Re: Searching Errors in a log file

Hi,

This may not be what you require, but I have used it alot. It will watch your log files and email you whenever it finds any security violations or errors. YOu can tailor it to watch for certain words - like "error"

http://sourceforge.net/projects/logcheck/

Rory R Hammond
Trusted Contributor

Re: Searching Errors in a log file


Jim,

Like all programming problems there are many tools with many solutions. My intent was to show how you can use grep like expressions inside of awk The tolower/toupper function inside the if statement is novel approach, along with the regular expression syntax. Of course, I made the fatal error of assuming that the only type of white space is a "space". Which in the syslogs I looked at was true.

a partial solution:

awk -F: '
(tolower($0) ~ /[ \t]error|[ \t]warn|[ \t]alert|[ \t]critical|[ \t]abort|[ \t]invalid/ ) {
print "\n" $0
for (i=1;i < 6;i++)
{
getline
print $0
}
} ' /var/adm/syslog/syslog.log

To bad I could not get the following to work:
(tolower($0) ~ /[:space:]error|[:space:]warn|[:space:]alert|[:space:]critical|[:space:]abort|[:space:]invalid/)

Rory
There are a 100 ways to do things and 97 of them are right