topic Re: hpux awk /time / average in Operating System - HP-UX

hpux awk /time / average

Rasheed Tamton — Mon, 28 Feb 2005 04:57:33 GMT

Hello,

I have a huge ascii file which has thousands of below records:

12832939
JSBKSM2
MSVD
7
5
2005-02-25 00:31:00.0
2005-02-25 00:41:00.0
2005-02-25 04:31:00.0
3
BAS3
000000504800948

02382631
Request executed successfully.
0
2005-02-25 00:31:01.0
###
12832940
JSBKSM7
MSVD
9
5
2005-02-25 00:31:00.0
2005-02-25 00:42:00.0
2005-02-25 04:31:00.0
3
BAS3
000000505819382

02137368
Request executed successfully.
0
2005-02-25 00:31:02.0
###
12832167
JSBKSM3
XXJK
7
5
2005-02-25 00:28:16.0
2005-02-25 00:28:26.0
2005-02-25 04:28:16.0
3
BAS3
000000503529256

08923355
Request executed successfully.
0
2005-02-25 00:28:17.0

The file starts with the first line (12832939 as in the example) which is the first column/field of the raw/record and ends with the date/time (2005-02-25 00:31:01.0 – first raw-last line in the file above). Each block or raw is separated by three hashes ###.

I need to know
1. how many times the MSVD (third line) appears in the file
2. how many times the value 7 or 8 or 9 appears in the fourth line IF THE VALUE on on the third line (the line above) is MSVD
3. The lines 6 and 7 have date and time as below:
2005-02-25 00:31:00.0
2005-02-25 00:41:00.0
I need to substract the TIME (second column) on the 6th line from the 7th line (7-6) and make an average of the output (total result) of all the substracted result.

I cannot use perl or gawk. I have to use the awk which comes with the hpux.

Thanks
Rasheed.

Re: hpux awk /time / average

Gordon Morrison — Mon, 28 Feb 2005 06:05:41 GMT

Hi Rasheed,

1) grep MSVD filename | wc -l

2)
#!/bin/ksh
let recline=0
let MSVD789=0
while read line
do
if [[ $line = "###" ]] ; then
let recline=0
else
let recline+=1
if (( $recline == 3 )) ; then
if [[ $line = "MSVD" ]] ; then
MSVDrec="TRUE"
else
MSVDrec=""
fi
fi
if (( $recline == 4 )) ; then
if [[ $MSVDrec = "TRUE" ]] ; then
if [[ $line = "7" || $line = "8" || $line = "9" ]] ; then
let MSVD789+=1
fi
fi
fi
fi
done < testfile
echo "Number of MSVD records: \c"
grep MSVD testfile | wc -l
echo "Number of MSVD 7,8&9 records: ${MSVD789}"

3) That's a hard one. It could be made a lot easier if the time format was a single number (e.g. number of seconds since the start of the epoch {1 Jan 1970})
Let me think about it for a bit.

Re: hpux awk /time / average

Elmar P. Kolkman — Mon, 28 Feb 2005 06:11:43 GMT

Checkout attached script. Should do what you want.

Or help you on your way.

Good luck,

Elmar

Re: hpux awk /time / average

harry d brown jr — Mon, 28 Feb 2005 07:30:49 GMT

perl comes with HP-ux 11 and up, so why can't you use it?

live free or die
harry d brown jr

Re: hpux awk /time / average

Gordon Morrison — Mon, 28 Feb 2005 08:09:21 GMT

perls are nice, but ksh is a gem :o)

#!/bin/ksh
let recline=0
let MSVD789=0

get_time()
{
time=$2
hours=${time%%:*}
tmp=${time#*:}
minutes=${tmp%%:*}
tmp=${time##*:}
seconds=${tmp%.*}
decsecs=${time##*.}
}

calc_diff()
{
if (( $start_h == $end_h )) ; then
let diff_h=0
elif (( $end_h > $start_h )) ; then
let diff_h=${end_h}-${start_h}
elif (( $end_h < $start_h )) ; then
let diff_h=${end_h}+24-${start_h}
fi

if (( $start_m == $end_m )) ; then
let diff_m=0
elif (( $end_m > $start_m )) ; then
let diff_m=${end_m}-${start_m}
elif (( $end_m < $start_m )) ; then
let diff_m=${end_m}+60-${start_m}
let diff_h-=1
fi

if (( $start_s == $end_s )) ; then
let diff_s=0
elif (( $end_s > $start_s )) ; then
let diff_s=${end_s}-${start_s}
elif (( $end_s < $start_s )) ; then
let diff_s=${end_s}+60-${start_s}
let diff_m-=1
fi

if (( $start_ds == $end_ds )) ; then
let diff_ds=0
elif (( $end_ds > $start_ds )) ; then
let diff_ds=${end_ds}-${start_ds}
elif (( $end_ds < $start_ds )) ; then
let diff_ds=${end_ds}+10-${start_ds}
let diff_s-=1
fi
}

# Main

while read line
do
if [[ $line = "###" ]] ; then
let recline=0
else
let recline+=1
if (( $recline == 3 )) ; then
if [[ $line = "MSVD" ]] ; then
MSVDrec="TRUE"
else
MSVDrec=""
fi
fi
if [[ $MSVDrec = "TRUE" ]] ; then
if (( $recline == 4 )) ; then
if [[ $line = "7" || $line = "8" || $line = "9" ]] ; then
let MSVD789+=1
MSVDval=$line
fi
fi
if (( $recline == 6 )) ; then
get_time ${line}
let start_h=$(( hours ))
let start_m=$((minutes))
let start_s=$((seconds))
let start_ds=$((decsecs))
fi
if (( $recline == 7 )) ; then
get_time ${line}
let end_h=$hours
let end_m=$minutes
let end_s=$seconds
let end_ds=$decsecs
calc_diff
echo "MSVD value = ${MSVDval}"
echo "Runtime = ${diff_h}:${diff_m}:${diff_s}.${diff_ds}"
fi
fi
fi
done < testfile

echo "Number of MSVD records: \c"
grep MSVD testfile | wc -l
echo "Number of MSVD 7,8&9 records: ${MSVD789}"

Re: hpux awk /time / average

Gordon Morrison — Mon, 28 Feb 2005 11:02:25 GMT

Oops! I forgot about your averages. Try this:

#!/bin/ksh
let recline=0
let MSVD789=0
let tot_h=0
let tot_m=0
let tot_s=0
let tot_ds=0
let numMSVD=0

get_time()
{
time=$2
hours=${time%%:*}
tmp=${time#*:}
minutes=${tmp%%:*}
tmp=${time##*:}
seconds=${tmp%.*}
decsecs=${time##*.}
}

calc_diff()
{
if (( $start_h == $end_h )) ; then
let diff_h=0
elif (( $end_h > $start_h )) ; then
let diff_h=${end_h}-${start_h}
elif (( $end_h < $start_h )) ; then
let diff_h=${end_h}+24-${start_h}
fi

if (( $start_m == $end_m )) ; then
let diff_m=0
elif (( $end_m > $start_m )) ; then
let diff_m=${end_m}-${start_m}
elif (( $end_m < $start_m )) ; then
let diff_m=${end_m}+60-${start_m}
let diff_h-=1
fi

if (( $start_s == $end_s )) ; then
let diff_s=0
elif (( $end_s > $start_s )) ; then
let diff_s=${end_s}-${start_s}
elif (( $end_s < $start_s )) ; then
let diff_s=${end_s}+60-${start_s}
let diff_m-=1
fi

if (( $start_ds == $end_ds )) ; then
let diff_ds=0
elif (( $end_ds > $start_ds )) ; then
let diff_ds=${end_ds}-${start_ds}
elif (( $end_ds < $start_ds )) ; then
let diff_ds=${end_ds}+10-${start_ds}
let diff_s-=1
fi
}

# Main

while read line
do
if [[ $line = "###" ]] ; then
let recline=0
else
let recline+=1
if (( $recline == 3 )) ; then
if [[ $line = "MSVD" ]] ; then
MSVDrec="TRUE"
else
MSVDrec=""
fi
fi
if [[ $MSVDrec = "TRUE" ]] ; then
if (( $recline == 4 )) ; then
if [[ $line = "7" || $line = "8" || $line = "9" ]] ; then
let MSVD789+=1
MSVDval=$line
fi
fi
if (( $recline == 6 )) ; then
get_time ${line}
let start_h=$(( hours ))
let start_m=$((minutes))
let start_s=$((seconds))
let start_ds=$((decsecs))
fi
if (( $recline == 7 )) ; then
get_time ${line}
let end_h=$hours
let end_m=$minutes
let end_s=$seconds
let end_ds=$decsecs
calc_diff
echo "MSVD value = ${MSVDval}"
echo "Runtime = ${diff_h}:${diff_m}:${diff_s}.${diff_ds}"
let tot_h+=$diff_h
let tot_m+=$diff_m
let tot_s+=$diff_s
let tot_ds+=$diff_ds
let numMSVD+=1
fi
fi
fi
done < testfile

let av_h=${tot_h}/${numMSVD}
let av_m=${tot_m}/${numMSVD}
let av_s=${tot_s}/${numMSVD}
let av_ds=${tot_ds}/${numMSVD}

echo "Number of MSVD records: ${numMSVD}"
echo "Number of MSVD 7,8&9 records: ${MSVD789}"
echo "Average runtime: ${av_h}:${av_m}:${av_s}.${av_ds}"

#####
# End
#####

Re: hpux awk /time / average

Muthukumar_5 — Tue, 01 Mar 2005 05:10:35 GMT

With your example,

muthu # awk 'BEGIN {RS="###";sev=x=egt=nine=t1=t2=avg=0} { if($3=="MSVD") { x=x+1; if($4==7) { sev=sev+1;} else if ($4==8){ egt=egt+1}else if ($4==9){nine=nine+1}} split($7,a,":"); t1=a[1]*60*60+a[2]*60+a[3]; split($9,b,":"); t2=b[1]*60*60+b[2]*60+b[3]; avg=avg+t2-t1;} END { print "MSVD COUNT = "x;print "Seven After MSVD = "sev;print "Eight After MSVD = "egt; print "Nine After MSVD = "nine;print "AVG : "avg" Sec" }' testfile
MSVD COUNT = 2
Seven After MSVD = 1
Eight After MSVD = 0
Nine After MSVD = 1
AVG : 1270 Sec

let me know if changes needed.

HTH.

Re: hpux awk /time / average

Rasheed Tamton — Mon, 07 Mar 2005 04:55:11 GMT

Thanks to all who responded to my query. I was busy with other jobs and did not get enough time to test it.

Gordon, you did marvellous with the shell script. I did not think before that the shell will be enough to do this.

Elmar, you gave me good pointer with the script though the script had some bad chars and was not working for me.

Harry, I am still not comforable well with the perl yet. Sorry for that. I am trying to improve.

Muthu, I do not get the correct avg with the script.

I still open the thread, if in case, someone can give some brilliant idea with the awk to get the required result.
Thanks
Rasheed.