topic Re: Monitoring a Log File in Operating System - HP-UX

Monitoring a Log File

dude70_1 — Thu, 11 Dec 2003 13:01:28 GMT

Hi Guys,

I need to monitor at the end of a logfile using a korn script continuously for a particular phrase to occur. Like "Error Occured" then I have to read the whole line and write to another file. Can you guys provide me any help!

Thanks in advance!
Dude70

Re: Monitoring a Log File

Paula J Frazer-Campbell — Thu, 11 Dec 2003 13:05:53 GMT

Hi

Try

tail logfile | grep "error occured" > otherfile

As a start point

Paula

Re: Monitoring a Log File

Paula J Frazer-Campbell — Thu, 11 Dec 2003 13:09:33 GMT

A little more

Create a file called monitorlog containing:-

--------------cut-----------------
#!/bin/sh
tail logfile | grep error occured > otherfile
sleep 10
exec monirorlog
----------------cut--------------

this will check the file and write out the error to otherfile, sleep for 10 seconds and then restart itself.

Paula

HTH

Paula

Re: Monitoring a Log File

Steven E. Protter — Thu, 11 Dec 2003 13:11:42 GMT

Modify the attached script, point it to the log you want to monitor, change lbolt to error or whatever and don't forget to change the email address.

I will have to embarass you if I get an email from your system... :-)

SEP

Re: Monitoring a Log File

dude70_1 — Thu, 11 Dec 2003 13:12:53 GMT

Thanks Paula,

Thing is the log file is dumped with lots of errors some times. I don't want to read the same error again. I want to start from the point where I finished reading before.

Thanks.

Re: Monitoring a Log File

Michael Schulte zur Sur — Thu, 11 Dec 2003 13:13:41 GMT

Hi,

look here:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=293706

greetings,

Michael

Re: Monitoring a Log File

Paula J Frazer-Campbell — Thu, 11 Dec 2003 13:14:55 GMT

Neat script Steven,

I was just giving pointers, you gave the solution.

;^)

Paula

Re: Monitoring a Log File

Paula J Frazer-Campbell — Thu, 11 Dec 2003 13:16:06 GMT

Hi

Can you post an example of the log file?

Paula

Re: Monitoring a Log File

Michael Schulte zur Sur — Thu, 11 Dec 2003 13:17:12 GMT

Hi Dude70,

this is the script from the thread from Bryan:
tail -f log.txt | while read LINE
do
echo $LINE | grep "Error Occured" >> log2.txt
done

greetings,

Michael

Re: Monitoring a Log File

John Carr_2 — Thu, 11 Dec 2003 13:21:32 GMT

how about this

tail -f logfile | grep my_error

tail will keep waiting on the logfile and feed new lines to grep as they occur continuing until you kill the process.

John.

Re: Monitoring a Log File

Steven E. Protter — Thu, 11 Dec 2003 13:29:49 GMT

Thanks Paula,

That script has saved me from two production problems this year.

The kudos are better than points.

My script is run daily via cron. For your situation you'll want to run it probably several times an hour. If you lose networking, it obviously won't help much.

SEP

Re: Monitoring a Log File

dude70_1 — Thu, 11 Dec 2003 17:38:00 GMT

Thanks guys for the overwhelming response! It was very helpful and I appreciate it.

I have a small question here. If my script is waiting for the string "Error Occured" indefinitely, will it create any perfornmance problem in the system? Because I want to run this script 24 * 7 * 365.

Re: Monitoring a Log File

dude70_1 — Thu, 11 Dec 2003 17:42:47 GMT

Also If I want to stop this script running in the background how should I do it?

Thanks!

Re: Monitoring a Log File

Steven E. Protter — Thu, 11 Dec 2003 17:49:29 GMT

Your best bet on running my script after you modify it(ignore if I'm wrong) is to run it on a reasonable schedule with cron.

If your system totally halts, everyone is going to notice and the script is no good.

If you run it once a minute and an error occurs with a DNS server and you get hundreds of errors, you are going to have a mailbox problem. Its going to be very full.

If you really want continuous monitoring then you will probably want the script to set a flag after it detects and error. Say change a data file in /etc from 0 to 1. When it picks up errors it makes a decision about doing a duplicate email, and only tries that say once a day.

Running the script too often is likely to burn too much resources.

If the system is mission critical(mine are) and you can't even tolerate 10 seconds of non-resopnse, then run it regularly or consider other kinds of monitoring tools like scm.

The script I uploaded only runs once a day. Its looking for bad disk drives and if it finds one, it runs another program that checks out all the disks. I'm attaching that as a reference because its useful as well.

Good Luck, thanks for the compliments and the points.

SEP

Re: Monitoring a Log File

dude70_1 — Thu, 11 Dec 2003 17:59:30 GMT

Hi Steven

I am actually using the following script and it is just fine for my purpose(writing all errors to another log). I want it to run 24*7. If I run this in the background forever is it ok? If I want to kill it(running in the background) how do I do it?

Thanks.

*******
tail -f log.txt | while read LINE
do
echo $LINE | grep "Error Occured" >> log2.txt
done

**********

Re: Monitoring a Log File

Hein van den Heuvel — Fri, 12 Dec 2003 00:28:06 GMT

SEP> Modify the attached script, point it to the log you want to monitor, change lbolt to error or whatever and don't forget to change the email address.

SEP> 41228.sh
SEP> :
SEP> /usr/sbin/dmesg | grep lbolt > /tmp/checkwardware
SEP> errs=`wc -l /tmp/checkwardware | awk {'print $1'}`

nit picking.... since the tmp file is only used to count lines from,
How about replacing the above two lines form the suggested script by this one:

/usr/sbin/dmesg | errs=`grep -c lbolt`

fwiw,
Hein.

Re: Monitoring a Log File

Michael Schulte zur Sur — Fri, 12 Dec 2003 04:47:25 GMT

Hi Dude70,

I have tested it. This script takes absolutely no cpu resources whatsoever, while waiting for a line. Tail -f is like a programme waiting for an input from stdin and the user is drinking his coffee. ;-) So you can run it 24 hours per day without problems.

have fun here,

Michael

Re: Monitoring a Log File

Michael Schulte zur Sur — Fri, 12 Dec 2003 04:50:44 GMT

Hi Dude70,

If you do it manually, do a ps -ef | grep yourscriptname.
Then kill -9 pid.

have fun,

Michael

Re: Monitoring a Log File

dude70_1 — Fri, 12 Dec 2003 14:05:35 GMT

Thanks Michael!

Re: Monitoring a Log File

dude70_1 — Fri, 12 Dec 2003 21:00:50 GMT

Guys,

I have a small problem here!
I am trying to capture the line that has the "Error Occured" string

tail -f log.txt | while read LINE
do
var= `echo $LINE | grep "Error Occured"`
echo $var
done

Apart from assigning the line to the variable var, list of files in the working directory is also added to $var. What am I doing wrong here?