Awk help again

[Belinda Dermody]

I have a maillog like below, one entry from a couple hundred thousand per day. Each line starts with a date time stamp. All I want from the log is grab the HeaderTo= and the line before which is HeaderFrom and the line after HeaderSubject

Apr 17 03:11:29 guadeloupe smap[15162]: ACCEPTING MESSAGE - INFORMATION FOLLOWS
Apr 17 03:11:29 guadeloupe smap[15162]: HeaderFrom='"ZZZZZ YYYYYY" acoast.com>'
Apr 17 03:11:29 guadeloupe smap[15162]: HeaderTo=''
Apr 17 03:11:29 guadeloupe smap[15162]: HeaderSubject='Your Perfect Home - New L
istings for 4/17/2003'
Apr 17 03:11:29 guadeloupe smap[15162]: SMTP HostName='smtp.onlinehelp.com'
Apr 17 03:11:29 guadeloupe smap[15162]: SMTP IPAddress='99.9999.999.99'
Apr 17 03:11:29 guadeloupe smap[15162]: SMTP HELO='smtp.onlinehelp.com'
Apr 17 03:11:29 guadeloupe smap[15162]: SMTP MAIL From='ZZZZZZZZ@Seacoast.com'

[Pete Randall]

James,

How about something like this, with grep, instead:

# print 1 line of context before and after regexp, with line number
# indicating where the regexp occurred (similar to "grep -A1 -B1")
sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h


Pete

[John Poff]

Hi,

Fleet Fingered Pete has given you a solution in sed [I love those sed one liners!]. Here is a little Perl snippet that might work:

perl -ne 'print if /HeaderFrom/ .. /HeaderSubject/' maillog

JP

[James R. Ferguson]

Hi James:

Yet another (this time in 'awk'):

# WHO=james
# awk -v WHO=$WHO '/HeaderFrom=/ && $0~WHO,/HeaderSubject=/ {print $0}' filename

Regards!

...JRF...

[Belinda Dermody]

Pete, you almost got it, but the trouble is
I want the three lines that are HeaderTo, HeaderFrom and HeaderSubject.

The key is on HeaderTo for the regexp. Sometimes it is
HeaderTo='

'
or
HeaderTo='address'

any clues

[Belinda Dermody]

James Ferguson, you are the same. I need to key in on the
HeaderTo and then grab the line before and the line after.

The problem is the HeaderTo could be either
HeaderTo='

'
or
HeaderTo='address'

[John Meissner]

of course you could always hire an attractive "administrative assistant" and have her pull these lines out by hand while you practice your awk skills :)

Sorry... it's been a looooon week with a 19 hour planned outage this past saturday.... i needed the humor

[Belinda Dermody]

John I have been practicing my awk skills, I have a habit of trying many different ways and have been working on this since Monday, so I decided to come to the experts, which I guess your field is comedy...

[John Meissner]

James - I'm not busting on your skills. I have actually tried to figure this type of problem out myslef (stripping out a range in awk & sed) but I don't think i've been that successful in the past. I usually found other ways around this. in order to get a range in sed you can
sed 's/beginning/,/end/g'

[John Poff]

James,

Is the name you are looking for an e-mail address in the form of somename@somedomain.com, or are you just looking for somename? Does the HeaderTo= always have single quotes around the name?

The Perl example I gave doesn't look for the name at all. I'll try that again.

JP

[John Meissner]

and out of all my post - I think that was my second attempt at humor.... i'll keep practicing :)

[Rodney Hills]

How about this-

awk '/HeaderTo=/{ print prev; print $0; getline ; print $0; }/;{prev=$0}' maillog

This will hold the line before in "prev", when HeaderTo is found prev and current line is printed. Then a "getline" reads the next line and prints it.

HTH

-- Rod Hills

[James R. Ferguson]

Hi (again) James:

OK, try this:

#!/usr/bin/sh
awk -v WHO="HeaderTo=$1" '
{ if ( $0~WHO ) {print LINE;print $0;getline;print $0} else {LINE=$0}
}' filename
exit 0

...pass the argument to match the string that follows "HeaderTo=". Thus if the script were called 'my.sh' do:

# my.sh "'"

...or:

# mysh "james"

...etc., depending on the string following the equal sign.

Regards!

...JRF...

[Francisco J. Soler]

Hi James,
I attach you a couple of scripts, with the first one the address is embeded in the script itself, in the second one you must type the address in the command line.

- First script.

awk -f file.awk inputfile > outputfile

- Second script

awk -v address="XXXXXXXX@alum\\.colby\\.edu" -f file2.awk inputfile > outputfile

hope this helps.

Frank.

[Pete Randall]

Sorry James,

While I will always think that the forums are the greatest resource there is, I'm supposed to be starting my vacation (and Momma said NOW!). I would attach all the "Handy One Liners for Sed" that Princess Paula originally posted so you might be able to noodle through - but I'm at home, I can't get to my notes - AND MOMMA SAYS NOW!!!



Pete

[John Poff]

Pete,

I've got you covered. I keep my sed one liners handy at work. I'm at home now, but I found a recent copy from the author's web page. I've posted it as an attachment.

JP

P.S. I understand what it means when Momma says NOW! Enjoy your vacation. :)

[Pete Randall]

Thanks, John!

I knew someone would pick up the slack. I'll send you a post card!

Pete

[Belinda Dermody]

Thanks to all you guy's, I haven't tried all the new responses yet, but that is why I have always insisted to my manager to make sure whatever support we have for our HP system that I have the forum option available. I very seldom call any longer, The response from the site (might be slow getting to it from the internet once in a while) is the greatest and the best and the diversacation of answers and abilities are the best and I have responded that to the leaders anumber of times.

We support online communities for Universities and Orgs and one of our perks is a email forwarding capabilities and of course the upper management like daily numbers and stats.
the To address could have ' ' around the address or < > or '< >' so I would just like to grab the address within the special chars.

[James R. Ferguson · Accepted solution]

Hi (again) James:

Based on your last comments, if you could find the lines you want regardless of how they are bounded, using the script I supplied, thusly:

# my.sh "'*<*James>*'*"

...would return matches for:

HeaderTo=James
HeaderTo=
HeaderTo=''
HeaderTo='James'

Regards!

...JRF...

[James R. Ferguson]

Hi (again) James:

Since you're filtering mail logs, perhaps you like to extract (match) names case-insensitively, so that "James" is the same as "james". If so try this script with your data:

#!/usr/bin/sh
typeset -l WHO="HeaderTo='*<*${1}>*'*"
awk -v WHO=$WHO '
{ if (tolower($0)~WHO) {print LINE;print $0;getline;print $0} else {LINE=$0}
}' filename
exit 0

...now to extract any variation of "jAmEs":

./my.sh jAmEs

Note that you pass one argument representing the name to match.

Regards!

...JRF...

[Belinda Dermody]

Jim Ferguson, thanks a lot, I am getting exactly what I am looking for with a little tweaking.

As always guys thanks for all the responses, comic including and for your efforts and see you at the next request