Logic required.

[Archana1]

HI,
Looking for logic which can capture red color pattern and check the difference of time stamp, incase the difference is more than 5000 sec. Should put in log file.
Please advice ?

TIMESTAMP Pattern 1 from log file:

2009-07-08 02:13:36.832|Warning|PipelinePairNode for fraudCheckResultAdd ::

TIMESTAMP Pattern 2 from same log file:

2009-07-08 02:13:39.345|Warning|PipelinePairNode for fraudCheckResultInq ::

TIMESTAMP Pattern 2 – TIMESTAMP Pattern 1
(in the above example 02:13:39 – 02:13:36 = 3 seconds)

>5000 sec

[James R. Ferguson]

Hi:

How do we match "red color pattern"?

That aside, Perl would allow you to convert the date/time into Epoch seconds and then evaluate the difference to log the data you want.

Regards!

...JRF...

[Archana1]

apologies for color.
The script shd grep for fraudCheckResultAdd and fraudCheckResultInq , In case the time stamp is more than 5000 sec the should log into log file.

I dont know perl much..If you give the script I can try out by executing.

[Dennis Handly]

>The script should grep for fraudCheckResultAdd and fraudCheckResultInq, In case the time stamp is more than 5000 sec the should log into log file.

Do these come only in pairs and not interspersed? And you want to check the difference in timestamps between only these two?

[Hein van den Heuvel · Accepted solution]

No takers? That would be because the problem was so poorly defined.
Given that we are looking for pieces of string and have to do some math, my initial tool of choice would be perl.
Perl has helpers for date/time manipulations. A quick google will find several (Date::Manip, Date::Parse,...)
Check that out.

But here I am thinking that something simple will do.... are those times ever more than a day apart? If so then the below will NOT work.
Just grab the time (could even skip seconds) and add a day add a day worth of seconds if the difference is negative.
The code below uses a simple print.
You may want a 'system' call to mail or some such.

You may also want to clarify for yourself what to do if one of the target records is missing, or repeated.

Simple working sample, with minimal (imho) sanity checking below.
Stick in a file "check_inq_time.pl" and execute as
# perl check_inq_time.pl input.txt
Or embed in a shell script.

Hein


my $MAX_TIME_DIF = 2; #5000;
while (<>) {
next unless /\S+\s+(\d\d):(\d\d):(\d\d)/;
$t = $1*3600 + $2*60 +$3;
$t_add = -$t if /fraudCheckResultAdd/;
if (/fraudCheckResultInq/) {
die "Found INQ but no ADD" unless defined($t_add);
$t_inq = $t; # just a flag for now
$t_add += $t;
$t_add += 86400 if $t_tadd < 0; # Date wrap?
print "ADD/INQ Time difference $t_add is larger than $MAX_TIME_DIF\n" unless $t_add < $MAX_TIME_DIF;
}
}
die "Found no INQ line" unless defined($t_inq);

[James R. Ferguson]

Hi:

You could begin with something like this:

# cat ./myscript
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
my $mindelta = 0; #...change as necessary...
my ( $line1, $line2, $delta, $time1, $time2 );
sub gettime {
my ( $year,$mon,$day,$hour,$min,$sec ) =
unpack 'a4xa2xa2xa2xa2xa2', $_[0];
return timelocal( $sec,$min,$hour,$day,$mon-1,$year-1900 );
}
while (<>) {
chomp;
if ( m{fraudCheckResultAdd} ) {
$line1 = $_;
$time1 = gettime($_);
$line2 = '';
$time2 = 0;
}
elsif ( m{fraudCheckResultInq} && $time1 > 0 ) {
$line2 = $_;
$time2 = gettime($_);
if ( ($delta = $time2-$time1) >= $mindelta ) {
print "$line1\n", "$line2\n", "Delta = $delta\n";
}
$time1 = $time2 = 0;
$line1 = $line2 = '';
}
}
1;

...run as:

# ./myscript logfile > log.out

Regards!

...JRF...

[Archana1]

Hi James RF, Thanks for script.

Not sure where the problem is ? not showing any result in log.out although the time differance is more than 5000 seconds.

$ more app.log
2010-11-18 02:13:36.832|Warning|PipelinePairNode for fraudCheckResultAdd :: PipelinePairNode f
or fraudCheckAdd _response:: Generate native response for Tuxedo:: RESPONSE| Before Receipt Of
Final Fraud Decision: 1F2504E0-4F89-11D3-9A0C-0305E82C3301
2009-11-18 04:13:39.345|Warning|PipelinePairNode for fraudCheckResultInq :: PipelinePairNode f
or fraudCheckResultInq _response:: Generate native response for Tuxedo:: RESPONSE| Before Rece
ipt Of Final Fraud Decision: 1F2504E0-4F89-11D3-9A0C-0305E82C3301

$ ./myscript app.log > log.out
$ more log.out
$

Please correct me in execution..

[James R. Ferguson]

Hi (again):

> Not sure where the problem is ? not showing any result in log.out although the time differance is more than 5000 seconds.

That's because your difference is a *negative* value. Based on your original data, the "Inq" record is expected to be later in time than the "Add" record.

The second set of data you presented uses the year 2010 and *then* the year 2009.

Regards!

...JRF...

[James R. Ferguson]

Hi (again):

By the way, when you are happy with the answers you have received, please read the following about the points system and their assignments:

http://forums11.itrc.hp.com/service/forums/helptips.do?#28

Regards!

...JRF...

[Dennis Handly]

>JRF: That's because your difference is a *negative* value.

Perhaps the input needs to be sorted, or the absolute value used.

[James R. Ferguson]

Hi:

> Dennis: Perhaps the input needs to be sorted, or the absolute value used.

Of course. And if we had a better problem definition, the necessity or lack thereof would be clear.

Since Perl has an 'abs()' function one could change:

if ( ($delta = $time2-$time1) >= $mindelta ) {

..to:

if ( ($delta = abs($time2-$time1)) >= $mindelta ) {

...

However, a clearer problem statement eliminates the guessing.

Regards!

...JRF...

[Hein van den Heuvel]

The script I presented in an earlier reply worked just fine with the crappy sample data, as it ignored the date.
Grins,
Hein

[Archana1]

Hello JRF,

Thanks for your script its my typo in scrap log file. Apologies for same.

$ more log.out
2010-11-18 02:13:36.832|Warning|PipelinePairNode for fraudCheckResultAdd :: Pipe
linePairNode for fraudCheckAdd _response:: Generate native response for Tuxedo::
RESPONSE| Before Receipt Of Final Fraud Decision: 1F2504E0-4F89-11D3-9A0C-0305E
82C3301
2010-11-18 04:13:39.345|Warning|PipelinePairNode for fraudCheckResultInq :: Pipe
linePairNode for fraudCheckResultInq _response:: Generate native response for Tu
xedo:: RESPONSE| Before Receipt Of Final Fraud Decision: 1F2504E0-4F89-11D3-9A0C
-0305E82C3301
Delta = 7203


Is this expected in log.out

Iam glad that I started learning perl scripting now..

[James R. Ferguson]

Hi:

> Is this expected in log.out

Yes. I assumed that you wanted to log the pair of records that triggered meeting the condition (a time difference greater than or equal to '$mindelta' written to the output. I added the time difference too.

As I originally noted, you can run the script passing the input file as an argument and redirecting the STDOUT stream to a file:

# ./myscript logfile > log.out

Regards!

...JRF...

[Archana1]

Hi JRF,
One more concern, when I test with real log file below are the messages and log.out is empty.

$ ./myscript app > log.out
Use of uninitialized value in numeric gt (>) at ./myperl line 14, <> line 10.
Use of uninitialized value in numeric gt (>) at ./myperl line 14, <> line 20.
Use of uninitialized value in numeric gt (>) at ./myperl line 14, <> line 30.


Appreciate your time. Seeming script tweaking is required.

[James R. Ferguson]

Hi:

> when I test with real log file below are the messages and log.out is empty.

The output is null because there are no "fraudCheckResultAdd" records.

As for the warnings from Perl, I should have initialized the variables. Change:

my ( $line1, $line2, $delta, $time1, $time2 );

...to:

my ( $line1, $line2, $delta, $time1, $time2 ) = ( '', '', 0, 0, 0 );

...

Haste makes waste :-)

Regards!

...JRF...