awk help

[Cem Tugrul]

Hello,
i have a log file which contains thousands of records.i want to write a script which gives me
last 1 month's records...My log file keeps
the date information like;
yyyy/mm/dd
and example one of the record for my log file as below;
R MAN 954627 954632 2006/06/06 17:28:19 KSAD2565 /users/rvs/usrdat/KSAD2565 2953
so as you see the 5th colon is keeping date information..As a result,if i want to run the script today so this means i need records
from 2006/06/06--->today

All suggetions would be pointed...
:-)
Good Luck,

[Steven E. Protter]

Shalom Cem,

Two step process.

Data looks like this:
R MAN 954627 954632 2006/06/06 17:28:19 KSAD2565 /users/rvs/usrdat/KSAD2565 2953

You can start with grep actually.

grep "2006/06" filename > newfile

You may need a special character to support the slash.

You can do it with awk.

while read -r dataline
do
datevar=$(echo $dataline | awk '{print $5}'
done < logfile

All you need to do now is parse the date variable.

yearvar=$(echo $datevar | awk -F\/ '{print $1}'
monthvar=$(echo $datevar | awk -F\/ '{print $2}'
dayvar=$(echo $datevar | awk -F\/ '{print $3}'

Not precisely sure about the -F statment. I'm assuming the special charcater treatment is required. I don't have time to test.

SEP

[H.Merijn Brand (procura]

# perl -MDate::Calc=Delta_YMD -ne'BEGIN{@now=localtime;($Y,$M,$D)=($now[5]+1900,++$now[4],$now[3])}($y,$m,$d)=Delta_YMD(m{\s(\d+)/(\d+)/(\d+)\s},$Y,$M,$D);print if !$y' test.dta

prints the last year

# perl -MDate::Calc=Delta_YMD -ne'BEGIN{@now=localtime;($Y,$M,$D)=($now[5]+1900,++$now[4],$now[3])}($y,$m,$d)=Delta_YMD(m{\s(\d+)/(\d+)/(\d+)\s},$Y,$M,$D);print if !$y && !$m' test.dta

prints all from the last month

Enjoy, Have FUN! H.Merijn

[Cem Tugrul]

Steven,

My 7th colon is the filename and changes...
As i said this file is log file which contains recieved files to my hp-ux so
i have to find out for some specific filenames but it does not matter i may grap it...the problem is getting records which belongs from last month's--->today
Anyway,i will test it...

Procura,

perl -MDate::Calc=Delta_YMD -ne'BEGIN{@now=localtime;($Y,$M,$D)=($now[5]+1900,++$now[4],$now[3])}($y,$m,$d)=Delta_YMD(m{\s(\d+)>
Can't locate Date/Calc.pm in @INC (@INC contains: /opt/perl5/lib/5.00502/PA-RISC1.1 /opt/perl5/lib/5.00502 /opt/perl5/lib/site_perl/5.005/PA-RISC1.1 /opt/perl5/lib/site_perl/5.005 .).
BEGIN failed--compilation aborted.

i think i made a mistake while running it

[Cem Tugrul]

baan01:/baan/bse/etc#perl -v

This is perl, version 5.005_02 built for PA-RISC1.1

Copyright 1987-1998, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5.0 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'. If you have access to the
Internet, point your browser at http://www.perl.com/, the Perl Home Page.

[H.Merijn Brand (procura]

Ah, oh, that is a rather old perl :)
And the problem in this case is not perl itself, but the fact that the module Date::Calc is not installed.

Would you consider a perl upgrade? (1),(2)
Would you like to install Date::Calc on the existing perl? (3),(4)
Or do you want a different solution altogether?

1) http://mirrors.develooper.com/hpux/#Perl
2) http://mirrors.develooper.com/hpux/downloads.html
3) http://search.cpan.org/~stbey/Date-Calc-5.4/
4) http://search.cpan.org/CPAN/authors/id/S/ST/STBEY/Date-Calc-5.4.tar.gz

Enjoy, Have FUN! H.Merijn

[Cem Tugrul]

Procura,
Once upon a time you again helped me about perl&inst&upgrade...it was really helpful.
i can think upgrade perl or install the
date program but not now so is there any
other solution to manipulate with awk?

[spex]

Cem,

First, use Clay's Date Hammer script (attached) to get the date a month in the past in the same format as your logfile. Then have awk output the first line in the logfile containing that date, along with all successive lines:

D=./caljd.sh -S "/" -y $(./caljd.sh -p 30)
awk "/${D}/,EOF" ${logfile}

If you only want certain lines returned by awk, pipe the output through grep.

PCS

[Peter Nikitka]

Hi,

an awk solution seems possible to me.
If we assume the file is ordered by date, we can output from the first pattern found without further checking:

awk -v ym=$(date +%Y/%m/%d) 'BEGIN {split(ym,z,"/"); if(z[2]+0 == 1) {z[1]--;z[2]=12}
else z[2]--
lookfor=sprintf("%s/%02d/",z[1],z[2])}
$5 ~ "^"lookfor {split($5,dd,"/");if (dd[3]+0 >= z[3]+0) doit=1}
doit' logfile


If the date strings in logfile are not written sequentially you have to
- define a lookfor2 in BEGIN containing the original year and month, e.g.
...
BEGIN {split(ym,z,"/");
lookfor2=sprintf("%s/%02d/",z[1],z[2])
if(z[2]+0 == 1) {...

- decide in each match with 'lookfor' and 'lookfor2'

I'm shure you can do this by yourself.

mfG Peter

[Mel Burslan]

Cem,

If you are looking for an awk one-liner, this posting is not it. And in my opinion, what you are trying to accomplish, requires a little more than a one-liner.

If you do not have it already, get Clay Stephenson's caljd.sh script from procura's web page http://mirrors.develooper.com/hpux/ (scroll to the bottom for the download links for both shell and perl versions. I personally use .sh version)

if you run caljd.sh without any arguments, you will get today in Julian date format, something like :

# ./caljd.sh
2453923
#

if you happen to run it like this

# caljd.sh 2453923
07 06 2006
#

so it basically toglles the julian format to regular date format. Nice thing about Julian date, you can add/subtract from that one.

I think you already figured out where I am going with this. All you need to do is something like this


todayjulian=`./caljd.sj`
monthlength=30 # modify it as you see fit to 28, 29 or 31

s=0

while [ $s -lt $monthlength ]
do
(( day=$todayjulian-$s ))
datestring=`./caljd.sh $day`

yr=`echo $datestring | cut -d" " -f3`
mo=`echo $datestring | cut -d" " -f2`
dy=`echo $datestring | cut -d" " -f1`

mydate=$yr"/"$mo"/"$dy

grep "$mydate" mylogfile >> last_month_log
# you can also do an awk {'print $5'} and compare it to $mydate if you prefer

(( s=$s+1 ))
done


Caveat emptors: It is not the most efficient way to do it and it is not tested, but as I follow your progress here, you can figure out the logic as a well experienced hp-ux person and modify it to your desires.

Kolay gelsin...