script to remove line

[haeman]

I have a log file ( event_log ) , there are many process write the word "testing error" to it , if I want to have a script to regularly ( the interval should be as short as possible , may be 1 or 2 second ) delete the line that have the word "testing error" ( the best is delete the line when it was created ) , can advise what can i do , can provide the script? thx


#vi event_log

aa
testing error
bb
cc
testing error
dd
testing error
ee
ff
gg

after update , it should be changed to
aa
bb
cc
dd
ee
ff
gg

[Kenan Erdey]

Hi,

sed '/^testing error/ d' event_log

you can redirect the output to another file.

Kenan.

[Saravanan_6]

sed -e '/testing error/d' < event_log > new_event_log

HTH..

[Davis Paul]

Hi
You can create a script containing following lines and put that in cron.
#!/usr/bin/sh
cat event_log |grep -v testing >event_log.new

Thanks
Davis Paul

[haeman]

thx replies,

the command is work, but if I want the script run frenquently ( eg. every 1 sec ) , what can i do ? thx

[Kenan Erdey]

it isnt' nice to mention but:

>I have assigned points to 1 of 116 responses to my questions.

A.Clay and SEP get very angry for these cases :)

[Kenan Erdey]

if you install gnu sed from this location:

http://hpux.cs.utah.edu/hppd/hpux/Gnu/sed-4.1.5/

you can use -i option to edit in place. So no need to redirect.

Kenan.

[haeman]

thx reply,

the script sed '/^testing error/ d' event_log will output the result to screen , if I want to directly update the event_log , what can i do ? thx

[Davis Paul]

Hi
Create a script like following and execute.

#!/usr/bin/sh
while ((1 > 0))
do
$cat event_log |grep -v testing >event_log.new
mv event_log.new event_log
sleep 5
done

[haeman]

thx reply

Davis Paul's script can do that , but I think some data will be lost when the file is in movement , the sed method seems good , can advise how to use it to direct update the file ? thx

[Kenan Erdey]

i wrote before. you can use gnu sed.

[Hein van den Heuvel]

Nah. All wrong.

What you want to do to solve this kind of problem is to use a PIPE aka FIFO

Instead od the providers writing (appending) to a real file, have them use a fifo. Now have a slave reading the fifo all the time and filter the data before going to the output.

For example.

A perl script: event_log_filter.pl
-------------
use IO::Handle;
open (OUT, ">>event_log_real") or die "no OUT";
OUT->autoflush(1);
while (1){
open (IN, " while () {
print OUT unless /^testing error/;
}
print "are we done yet?\n";
}
------------

Now try in one window

# mkfifo event_log
# perl event_log_filter.p &
# tail -f event_log_real

And in an other window (or windows)

# cat > event_log
blah
blah blah
testing error
blah


Good luck!
Hein

[Dennis Handly]

>the interval should be as short as possible, may be 1 or 2 second)

I near panicked when I read this time requirement. You can't edit a file where other processes are trying to write to it. (Unless you are using fancy locking, etc.)

>can advise how to use it to direct update the file?

This will still lose data.

>Hein: All wrong.
>What you want to do to solve this kind of problem is to use a PIPE aka FIFO

Exactly. Or filter the stuff out just before you use it. (This wouldn't be helpful if the junk bloats the file too much.)

>Kenan: A.Clay and SEP get very angry for these cases :)

(It's called struck off. :-) (haeman's now up to 64!)

[haeman]

thx reply,

I think Hein van den Heuvel 's method is OK , but it is required to install module to the server , it is too complicate for me ,

I still think the command "sed '/^testing error/ d' event_log" is simple and meet my requirment , but I would like to ask how to direct update the file instead output to another file ?

please ignore the data lost issue .

thx

[James R. Ferguson]

Hi:

> I think Hein van den Heuvel's method is OK, but it is required to install module to the server, it is too complicate for me

That's not true. the 'IO::Handle' modules was released with Perl 5.00307. If, however, you would like to skip using it, for whatever reason, you can make the following change:

Instead of the line:

# OUT->autoflush(1);

...use these two statements:

# select OUT;
# $|++;

Regards!

...JRF...