'tail -f' piped with sed

[Markus Gerber]

Hello everybody,

I'm trying to trim some log files for better clarity for the operator. Therefore the logfiles shall only be altered on-the-fly, meaning I should not change the amount of log messages which are writen into the file. But I shall change the amount and look of the log messages which are printed out.

So I come to the idea to make a "tail -f | sed -f sed_command.file" command, but this command always ignores the latest log message. When a newer log message comes in, the older message apears, but now the new message doesn't show up.

I think that this error comes from the fact, that "tail -f" does not output a newline until a new message comes in.

Is it possibel to show up all log messages wihtout delay?

Thanks for your help in advance!

Regards, Markus

[James R. Ferguson]

Hi:

This is likely a buffer issue. If the messages (as you define "message") don't end in a newline ("\n") then they will be buffered until a newline flushes them.

Regards!

...JRF...

[Markus Gerber]

Hi,

yes I already thought about that.

Do you see a way to "manually" send a newline or do a flush to the sed buffer?

What would you recommend to make it possible?

Thanks a lot!

Markus

[James R. Ferguson]

Hi Markus:

OK, I can reproduce what you are seeing with 'sed'. If I use Perl I can defeat your problem. Post your 'sed' script if you like.

Regards!

...JRF...

[Markus Gerber]

Hi,

right now I'm using just a basic script for testing (attached)!

The comamnd I'm using:

"tail -f /clog/syslog/syslog.log | sed -f sed_command.file"

Thanks and Regards,

Markus

[James R. Ferguson · Accepted solution]

Hi (again) Markus:

# cat filter
#!/usr/bin/perl -p
s/(sapwas_main): Entering SGeSAP stop runtime steps .../SAP is beeing stopped .../;
s/(sapwas_main): Entering SGeSAP start runtime steps .../SAP is being started .../;
s/(sapwas_main): Leaving SGeSAP start runtime steps/SAP was successfully started/;
s/(sapwas_main): Leaving SGeSAP stop runtime steps/SAP was successfully stopped/;
s/ [A-Z][a-z][a-z] [1-3 ][0-9] [0-9][0-9]\:[0-9][0-9]\:[0-9][0-9] - Node \"s96hph0[1-4]\"://;

...run as you would expect:

# tail -f file | ./filter

Regards!

...JRF...

[James R. Ferguson]

Hi (again) Markus:

If you wish to continue to use 'sed', the GNU version offers the '-u' switch --- exactly what you need when handling input from something like 'tail -f'.

Regards!

...JRF...