topic Re: summary colums in shell script in Operating System - Linux

summary colums in shell script

Jairo Campana — Mon, 04 Feb 2008 19:40:04 GMT

Hi ,

2 questions:

1. I question, how to calculate sum of all average in each item total crc/err linkfail and loss sync for each port .
example :
for port 0: , total crcerr,linkfail,losssync
these fields change constantly in each sleep 60
port 1: ...

2. and when the field is different of 0 display average or diferent of 0 , diplay in standart oupout for each port
0:
1:
2:

It really helped me. Thank you.

columns 5,11,12(crc error, linkfail, losssync

attach log file

Re: summary colums in shell script

James R. Ferguson — Mon, 04 Feb 2008 20:30:49 GMT

Hi Jairo:

You can use 'awk' to perform the matching and arithmetic. For example, to output only the lines with a non-zero value in field-12, and print the average of the column's value at the end, you could do something like:

# mycommand | awk 'NR>3 {if ($12~/k/) {F12=$12*1024} else {F12=$12};SUM+=F12;if (F12>0) {print}};END{print SUM/(NR-3)}'

Notice that I accounted for values with a 'k' magnitude by multiplying the value by 1024. You would need to embellish this logical to handle values with 'm' (1024*1024). The header lines (1-3) are skipped, and of course, not counted in computing the average.

Regards!

...JRF...

Re: summary colums in shell script

Hein van den Heuvel — Tue, 05 Feb 2008 01:05:17 GMT

Jairo,

The desired output description is a little terse, and possibly suggests a misunderstanding of the data presented.

First, This is the output of 'portErrShow' for a Brocade SAn switch isn't it?
Please indicate so!
So it is possibly more a Storage forum question than HPUX isn't it?

Second, Best I know (and I may well be wrong!) those error rates are accumulated already. So you really want to subtract the prior value from the current value to see what happened this period.

Third, these error counters are really only relevant as a proportion to the actual traffic. Some might suggest that an error rate up to 0.5% of the traffic is still acceptable. There is not a very clear example of this in your output, but take port 11 and 12. Similar error counts, but 1000x more traffic for #11, so the fewer errors on #12 may still be more significant.

Fourth, there may have been an 'episode' since repaired / dissapeared causing a high static values. So you may want a rolling average over an hour or over 24 hours, requiring some sort of memory in the calculations, possibly spooling all to disk and reporting from there.

Finally, try to get a timestamp in the output. It may be blatantly obvious to you today that this was done at 1 minute intervals, but that is not clear to a processing script. And just in case the report get's stalled for some time it would be nice to eb able to 'see' that in averaging.

Hopefully the questions above help you define the output better!

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

Re: summary colums in shell script

Jairo Campana — Mon, 11 Feb 2008 19:31:15 GMT

THANKS JAMES:

yes Its work , now need analisys the result and print the result, howto
example:
in the while true the values change each "sleep 5"
while true
do
cat logwcrc|awk '{print $5}'
sleep 2
done

I need in each value for the row one condition in while true read , compare if value > 10, assign en variable SUM, in the following evaluation compare the last value store in the variable if SUM > 30 then report $SUM, else return evaluation or finish

the value in each row to the result evaluate , print to a flank example

cat logwcrc|awk '{print $5}'
# cat logwcrc|awk '{print $5}'

1
0
0
...
15 SUM=20
0
0
0
0
0
0
0

Re: summary colums in shell script

Jairo Campana — Mon, 11 Feb 2008 19:53:48 GMT

I create the following script :
#!/usr/bin/sh
for i in `cat logwcrc|awk '{print $5}'|sed '/^$/d'|grep -v too|grep -v shrt`
do
if [ $i -gt 10 ]
then
SUM=$i
if [ $SUM -gt 20 ]
then
echo "CRC ERROR ACUMULADO de $SUM"
fi
fi
done

Re: summary colums in shell script

Jairo Campana — Mon, 11 Feb 2008 20:07:06 GMT

I cant use to
while read i j
do
...

done < $LIST

Re: summary colums in shell script

Hein van den Heuvel — Mon, 11 Feb 2008 20:17:30 GMT

Jairo,

Please READ, and study, the exampled provided.
And please spend a few minuted with the man pages for AWK.

For example, in the last post you used:

awk '{print $5}'|sed '/^$/d'|grep -v too|grep -v shrt

This last 3 pipes appear to perform an inneficient, and error prone, removal of the first 3 lines from awk.

But our friend James already spend some time showing us how to do this elegantly, fast, and safe. All that 'crud' can be replaced by:

awk 'NR>3 {print $5}'

And awk can just read from a file. No cat needed:

awk 'NR>3 {print $5}' logwcrc

And awk will happily do the extra condition testing 'for free'

awk 'NR>3 && $5>20 {print "CRC ERROR ACUMULADO de " $5} logwcrc

Enjoy!
Many happy learnings.
Hein.