Operating System - HP-UX
1827835 Members
9097 Online
109969 Solutions
New Discussion

Re: Script help needed to parse /var/mail files

 
SOLVED
Go to solution
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

I think I know what the problem is. When I am getting the blanked out messages, it seems to be because the second message is similar to the one before it (ie. the From, To, and Subject are the same). I'm thinking the script is not resetting. How can I overcome this (if this is the problem)? Thanks again
Hein van den Heuvel
Honored Contributor

Re: Script help needed to parse /var/mail files

Like I said, I do not know and do not really want to know the mail file format.
What defines a new messages start or an end?
It _seemed_ to me like it is to open:
--GAA00598.1215080246/here.com
and close:
--GAA00598.1215080246/here.com--

So that's what the script looks for.
When I append the above text chunk to a mail file, it doe s nothing. But when I also add the missing (to the best of my limited knowledge) end line as "--GAA00598.1215080246/here.com--", then it reports fine for me. Dunno whether that line is not there due to a cut & past problem or whether is really does not need to be there and I need a better trigger like "EOF" or a next, different line like "--GAA00598.1215080246/here.com"

Maybe it well be more clear when appending a (chunk of) a real mail file as example.

Here is a minor re-hash to trigger on a new ID or EOF or --OLD/ID--

Hein.
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Here is a "sample" of two messages in from a user's mail. Hope this is useful. Thanks again for the help!!!
Eric Lipede
Frequent Advisor

Re: Script help needed to parse /var/mail files

Hi

have a look at this thread - it strikes me that the response do exactly what you would like (ish)

http://forums12.itrc.hp.com/service/forums/questionanswer.do?threadId=1245624
Quod sum eris
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Nope...not exactly but thanks.
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Minus that one little bug, Hein's script works GREAT!!! Hein is the MAN!!!
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Hey gang,

I've been using the script that I got from here with lots of success....too much success unfortunately. I am having requests to change the look of the output for more tracking capability. I have attached the the script being used.

Here is how the output looks now...

Final-Recipient: RFC822; baduser@here.com
Action: failed
Status: 5.1.2
Subject: Returned mail: Host unknown (Name server: here.com: host not found)
Date: Thu, 10 Sep 2009 11:26:06 -0400
From: Me
To: You@there.com
Subject: TESTING

Question: How can this be modified to output information as such (changes are in parenthesis)...

Final-Recipient: baduser@here.com (notice nothing after "Final-Recipient")
Action: failed
Status: 5.1.2
Problem: Returned mail: Host unknown (Name server: here.com: host not found)(Word "Subject" is replaced with "Problem")
Date: Thu, 10 Sep 2009 11:26:06 -0400
From: Me
To: You@there.com
Subject: TESTING

Thanks for your help.
James R. Ferguson
Acclaimed Contributor

Re: Script help needed to parse /var/mail files

Hi:

Change line-46 from:

$recipient = $_ if /^Final-Recipient:/;

To:

($recipient=$_)=~s/(^Final-Recipient:)(?:.+;)(.+)/$1$2/ if /^Final-Recipient:/;

Change line-49 from:

$real_subject = $_ if /^Subject:/;

To:

($real_subject=$_)=~s/^Subject:/Problem:/ if /^Subject:/;

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Hi James,

The modifications you suggested produced a bunch of digits instead of the desired output. Any thoughts? Thanks again.
James R. Ferguson
Acclaimed Contributor

Re: Script help needed to parse /var/mail files

Hi (again):

The modifications I suggested, based on my understanding of your requires; using Hein's script attached by you on Sept 14 @ 19:44; together with the data attached on
Jul 8 @ 00:38, seem to work.

If I execute Hein's version and collect the output in "myout.old" and then execute my modified version and collect the ouput in "myout.new"; and do:

# diff /tmp/output.old /tmp/output.new
2c2
< Final-Recipient: RFC822; recipient@here.com
---
> Final-Recipient: recipient@here.com
9c9
< Subject: Report
---
> Problem: Report

The integrated modification's to the script make it look like:

#!/usr/bin/perl
use strict;
use warnings;
my $step=0;
my $id=0;
my $old=1;
my ($problem, $recipient, $action, $status, $subject, $date, $from, $to, $real_subject ) ;
while (<>) {
#debug print "$step $.\n" if $step != $old;
#debug $old = $step;

if (/^--(\S+\/\S+?)(-?-?)\s?$/) {
if ($1 ne $id) { # New ID? Clear all memory
$id = $1;
$step =0;
$date = q()."\n";
$from = q()."\n";
$to = q()."\n";
$recipient = q()."\n";
$action = q()."\n";
$status = q()."\n";
$real_subject = q()."\n";
} else {
if ($2) { # Found closing marker
print $problem, $recipient, $action, $status, $subject, $date, $from, $to, $real_subject, "\n";
$step = 0; # Done. Start again.
}
}
}

if (0==$step) {
$subject = $_ if/^Subject:/;
# next unless /^[ -]+.*permanent fatal.*-\s?$/;
next unless /^[ -]+.*fatal error.*-\s?$/;
$step++;
}
elsif (1==$step) {
next unless /^[ -]+Transcript.*-\s?$/;
$problem = <>;
$step++;
}
elsif (2==$step) {
$date = $_ if /^Date:/;
$from = $_ if /^From:/;
$to = $_ if /^To:/;
($recipient=$_)=~s/(^Final-Recipient:)(?:.+;)(.+)/$1$2/ if /^Final-Recipient:/;
$action = $_ if /^Action:/;
$status = $_ if /^Status:/;
($real_subject=$_)=~s/^Subject:/Problem:/ if /^Subject:/;
}
}

...the changed lines being line-46 and line-49 as previously noted.

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Hi James,

Your modification worked perfectly!!! Thank you for your help!!! I do have one more question for you....

When the script is going through a message in /var/mail and there are multiple failures in a single email, the script will only show the first failed recipient but fail to catch any of the others. Is there a way to modify the script so that it will get every failed recipient? Thanks
James R. Ferguson
Acclaimed Contributor

Re: Script help needed to parse /var/mail files

Hi (again):

> Is there a way to modify the script so that it will get every failed recipient?

If you mean like this (using) you data as attached on Jul 8 @ 00:38 :

... while talking to here.com.:
Final-Recipient: recipient@here.com
Action: failed
Status: 5.1.1
Subject: Returned mail: User unknown
Date: Thu, 3 Jul 2008 06:17:24 -0400
From: sender
To: recipient@here.com
Problem: Report

... while talking to here.com.:
Final-Recipient: recipient@here.com
Action: failed
Status: 5.1.1
Subject: Returned mail: User unknown
Date: Thu, 3 Jul 2008 06:17:26 -0400
From: sender
To: recipient@here.com
Problem: Report

...then use this slightly modified version:

#!/usr/bin/perl
use strict;
use warnings;
my $step=0;
my $id=0;
my $old=1;
my ($problem, $recipient, $action, $status, $subject, $date, $from, $to, $real_subject ) ;
while (<>) {
#debug print "$step $. $_\n" if $step != $old;
#debug $old = $step;
my ($start, $stop);
if (($start,$stop) = /^--(\S+\.\d+)\S+?(-?-?)\s?$/) {
if ($start ne $id) { # New ID? Clear all memory
#debug print "<<< $. $_";
$id = $start;
$step =0;
$date = q()."\n";
$from = q()."\n";
$to = q()."\n";
$recipient = q()."\n";
$action = q()."\n";
$status = q()."\n";
$real_subject = q()."\n";
} else {
if ($stop) { # Found closing marker
print $problem, $recipient, $action, $status, $subject, $date, $from, $to, $real_subject, "\n";
$step = 0; # Done. Start again.
}
}
}
if (0==$step) {
$subject = $_ if/^Subject:/;
# next unless /^[ -]+.*permanent fatal.*-\s?$/;
next unless /^[ -]+.*fatal error.*-\s?$/;
$step++;
}
elsif (1==$step) {
next unless /^[ -]+Transcript.*-\s?$/;
$problem = <>;
$step++;
}
elsif (2==$step) {
$date = $_ if /^Date:/;
$from = $_ if /^From:/;
$to = $_ if /^To:/;
($recipient=$_)=~s/(^Final-Recipient:)(?:.+;)(.+)/$1$2/ if /^Final-Recipient:/;
$action = $_ if /^Action:/;
$status = $_ if /^Status:/;
($real_subject=$_)=~s/^Subject:/Problem:/ if /^Subject:/;
}
}

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

I have had to modify the script to look for different things (it is attached). I am experiencing two problems....

1. If there are multiple bounce-backs in a single message, the script is only grabbing one of them (mailbox2 should produce multiple entries)

2. It is getting the subject and real-subject incorrectly (see results below...) I'm sure it is from the modifications I tried to make.

This is the output I get when I run this script against mailbox1 and mailbox2 files (see next two posts for the files)...


Final-Recipient: recipient1@there.com
Subject: TEST
Action: failed
Status: 5.1.1
Problem: TEST
Date: Mon, 21 Sep 2009 15:00:23 -0400
From: Me
Last-Attempt-Date: Mon, 21 Sep 2009 15:00:25 -0400 (EDT)


Final-Recipient: recipient3@baddomain.com
Subject: TESTING
Action: failed
Status: 5.1.2
Problem: TESTING
Date: Mon, 21 Sep 2009 19:10:47 -0400
From: Sender
Last-Attempt-Date: Mon, 21 Sep 2009 19:10:47 -0400 (EDT)

Is there a way to mod the script to get all of the entries and separate the subject and problem? Thanks again
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

mailbox1 file from /var/mail
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

mailbox2 file from /var/mail
James R. Ferguson
Acclaimed Contributor

Re: Script help needed to parse /var/mail files

Hi:

OK, try the attached modifications. I have reformatted the original script for readability, too.

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Hi James,

Thank you again. However, I'm getting the following output from the latest modified script:


The original message was received at Mon, 21 Sep 2009 15:00:23 -0400 (EDT)
Final-Recipient: recipient1@there.com
Subject: Returned mail: User unknown
Action: failed
Status: 5.1.1
Date: Mon, 21 Sep 2009 15:00:23 -0400
From: Me
Problem: TEST
Last-Attempt-Date: Mon, 21 Sep 2009 15:00:25 -0400 (EDT)
The original message was received at Mon, 21 Sep 2009 19:10:47 -0400 (EDT)
Final-Recipient: recipient3@baddomain.com
Subject: Returned mail: Host unknown (Name server: baddomain.com: host not found)
Action: failed
Status: 5.1.2
Date: Mon, 21 Sep 2009 19:10:47 -0400
From: Sender
Problem: TESTING
Last-Attempt-Date: Mon, 21 Sep 2009 19:10:47 -0400 (EDT)
Last-Attempt-Date: Mon, 21 Sep 2009 19:10:47 -0400 (EDT)
Last-Attempt-Date: Mon, 21 Sep 2009 19:10:47 -0400 (EDT)

The subject and problem lines are reversed. Also it is not showing all of the recipient errors in the mailbox2 file. Is there a way to fix this? Thank you very much!!!
James R. Ferguson
Acclaimed Contributor

Re: Script help needed to parse /var/mail files

HI (again):

> The subject and problem lines are reversed. Also it is not showing all of the recipient errors in the mailbox2 file.

OK, then if this still isn't what you want, show me a sample output (fixed up manually).

That said, see the attachment for the latest version.

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

Hi James,

A perfect output (using these mailbox files as examples), would be something similar to this (please note that the "Original-Envelope-Id" entry is something that will need to be filtered for in some bounce-back messages):


From: Me
Date: Mon, 21 Sep 2009 15:00:23 -0400
Final-Recipient: recipient1@there.com
Subject: TEST
Action: failed
Status: 5.1.1
Problem: Returned mail: User unknown


From: Sender
Date: Mon, 21 Sep 2009 19:10:47 -0400
Final-Recipient: recipient1@baddomain.com
Final-Recipient: recipient2@baddomain.com
Final-Recipient: recipient3@baddomain.com
Subject: TESTING
Action: failed
Status: 5.1.2
Problem: Returned mail: Host unknown (Name server: baddomain.com: host not found)

Thanks again for your help!!!!
James R. Ferguson
Acclaimed Contributor
Solution

Re: Script help needed to parse /var/mail files

Hi (again):

See the attachment.

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

James, you are THE man!!! That works wonderfully!!! Thank you VERY much for all of your help!!!
James R. Ferguson
Acclaimed Contributor

Re: Script help needed to parse /var/mail files

Hi:

Although I'm glad to have helped, most of the credit in this case goes to Hein. After all, I only modified his script...

If he will chime in, you owe him...

Regards!

...JRF...
TheJuiceman
Super Advisor

Re: Script help needed to parse /var/mail files

You are right. A HUGE "thank you" to you both!!! You guys are terrific!!!