topic Re: greping some patterns between some timeframes in Operating System - HP-UX

greping some patterns between some timeframes

Shivkumar — Mon, 30 Mar 2009 23:10:18 GMT

Hello,

I want to grep some error messages but the error message name is not known to me from some logs.
Log has the timing pattern as
2009-03-29 02:31:01.

Say i want to grep messages for particular day 2009-03-29. The messages should be between time frame 02:31:01-02:39:59. Between this said timeframe i want grep say "errors or exception" like patterns.

Can someone suggest some best way to do it.
I know some crude way but just curious to know if i can get some good tricks from seasoned Admins of this forums.

Thanks,
Shiv

Re: greping some patterns between some timeframes

Hein van den Heuvel — Tue, 31 Mar 2009 01:39:41 GMT

Grep isn't gonna cut that.

A simple AWK range expression seems ok like:
$ awk '/03-29 02:33/ , /02:36:01./ {if ( /error/){print}}' tmp.txt
But it only works if you can garantue each minute to be in the file.

A more complex rang might be tempting like:
'$2>"02:34:00." , $2>"02:37:00."'
That will get close, but the none-date lines like 'error' muck that up.

So you have to create your own 'zone' looking for a time range. For example:

$ awk '/2009/ && $2>"02:32:00."{zone=1} /2009/ && $2>"02:40:00."{zone=0} zone && /error/'

Hope this helps,
Hein.

Re: greping some patterns between some timeframes

Mark McDonald_2 — Tue, 31 Mar 2009 01:42:01 GMT

grep "2009-03-29 02:3"

is very nearly what you are after.

you can use regular expressions in grep too, so include [1-5] after the 02:3 to only get times from 02:31 to 02:35.

I'm pretty sure all of these will work:
^ (Caret) = match expression at the start of a line, as in ^A.
$ (Question) = match expression at the end of a line, as in A$.
\ (Back Slash) = turn off the special meaning of the next character, as in \^.
[ ] (Brackets) = match any one of the enclosed characters, as in [aeiou]. Use Hyphen "-" for a range, as in [0-9].
[^ ] = match any one character except those enclosed in [ ], as in [^0-9].
. (Period) = match a single character of any value, except end of line.
* (Asterisk) = match zero or more of the preceding character or expression.
\{x,y\} = match x to y occurrences of the preceding.
\{x\} = match exactly x occurrences of the preceding.
\{x,\} = match x or more occurrences of the preceding.

Re: greping some patterns between some timeframes

Bill Hassell — Tue, 31 Mar 2009 01:55:55 GMT

Once you can return the right date/time range, searching for errors and exceptions is easy. grep has a -e option that can be used many times and is more versatile than egrep and extended regex strings. Here's a way to search thru syslog for problems:

grep -ie err -e fail -e exception -e nospace -e disable -e lockout -e exceed -e invalid -e attempt -e refuse /var/adm/syslog/syslog.log

Re: greping some patterns between some timeframes

Hein van den Heuvel — Tue, 31 Mar 2009 02:09:59 GMT

Shiv,

Does every line have a date stamp, or is it more like and Oracle Alert log which looks like:
:
Date & time.
text
Date & other time.
other text
:

If there is a timestamp everywhere, then Mark's suggestion will work fine for narrow time ranges. It gets a little coarse when looking for say 02:29 - 02:31.

And forgot about my remark:
>> A more complex rang might be tempting like:
'$2>"02:34:00." , $2>"02:37:00."'

That's nonsense. The first clause keep on becoming true, so it is not a suiteable for range. Brain-fart. I meant a range like:

$ awk '$2>"02:30:99." && $2<"02:39:99." && /error/' tmp.txt

But again... only if each line has a time & day which I assumed it did not.

(yeah I know, there are not 99 seconds in a minute. That's just to amplify that awk treats $2 as a piece of strings, not a time )

Hein.