awk/perl help.

[RAC_1]

Putting it here to get more responses. I need perl/awk help from you guys. simple, but proving to be difficult for me.

HP-UX xxxyyy B.11.11 U 9000/800 01/05/06

03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c1t2d0 11.88 0.50 21 97 4.35 10.33
c2t2d0 10.89 0.50 19 89 4.50 7.35
c4t3d0 0.99 0.50 1 16 2.89 16.59
c4t4d0 0.99 0.50 1 16 2.90 12.09
c4t5d0 0.99 0.50 1 16 1.53 9.00
c5t5d0 0.99 0.50 1 16 1.54 11.23
c12t0d3 0.99 0.50 7 453 6.54 1.62
c13t0d4 100.00 0.50 1 2 4.64 0.49
c12t1d4 0.99 0.50 7 137 3.20 1.13
c12t0d0 1.98 0.50 33 507 5.08 0.60


I have following output (from sar -f) I need to get output based on following criteria.

Print a record if (last second last column (avwait) is more than aserver (last column))
further, %busy is more than 60.

I tried following, but not getting what I want.
cat "xxx" |Average"|awk '{if (($1~":") && ($3>60) && $((NF-1))>$NF) print $0;else if (($1!~":") && ($2>60) && $((NF-1))>$NF) print $0;else print "" ;}'


I should get only following lines.

03:32:24 device %busy avque r+w/s blks/s avwait avserv
c13t0d4 100.00 0.50 1 2 4.64 0.49

[RAC_1]

where are those scripting wizards??

[Peter Godron]

RAC,
is the output not correct?
The only record with %busy > 60 is record 8 (c13t0d4)
I copied the data into file xxx ,copied your statement into a sh and changed
the beginning to
cat xxx | awk ...
and got
c13t0d4 100.00 0.50 1 2 4.64 0.49

What are you getting as output ?

[H.Merijn Brand (procura]

Simple :) :)

lt09:/home/merijn 103 > cat xx.txt
03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c1t2d0 11.88 0.50 21 97 4.35 10.33
c2t2d0 10.89 0.50 19 89 4.50 7.35
c4t3d0 0.99 0.50 1 16 2.89 16.59
c4t4d0 0.99 0.50 1 16 2.90 12.09
c4t5d0 0.99 0.50 1 16 1.53 9.00
c5t5d0 0.99 0.50 1 16 1.54 11.23
c12t0d3 0.99 0.50 7 453 6.54 1.62
c13t0d4 100.00 0.50 1 2 4.64 0.49
c12t1d4 0.99 0.50 7 137 3.20 1.13
lt09:/home/merijn 104 > perl -nae'$F[-2]>=$F[-1]&&$F[2]>=.6 and print' xx.txt
lt09:/home/merijn 105 > perl -nae'$F[-2]>=$F[-1]&&$F[2]>=.5 and print' xx.txt
c12t0d3 0.99 0.50 7 453 6.54 1.62
c13t0d4 100.00 0.50 1 2 4.64 0.49
c12t1d4 0.99 0.50 7 137 3.20 1.13
lt09:/home/merijn 106 >

Enjoy, Have FUN! H.Merijn

[H.Merijn Brand (procura]

Small change, as I took the headers of the first line to the meaning of the columns

lt09:/home/merijn 102 > perl -nae'$F[-2]>=$F[-1]&&$F[1]>=60 and print' xx.txt
c13t0d4 100.00 0.50 1 2 4.64 0.49
lt09:/home/merijn 103 >

Enjoy, Have FUN! H.Merijn

[RAC_1]

Peter,

I am not getting correct output with my awk command. I also need to get following lines.

00:00:00 device %busy avque r+w/s blks/s avwait avserv

and first line in output.

these two lines have timestamp, which is required to associate that data with time when it happened.

Procura,

How to include above requirement in perl statement?

[Hein van den Heuvel]

perl -nae'/avserv/ || $F[-2]>=$F[-1]&&$F[1]>=60 and print' x


or

awk '/avserv$/; /^c/{if ($2>60 && $(NF-1) > $NF)print}' x

but...

What is that "03:32:25" on the line with "c1t2d0"?
Is that for real or cut&paste thing?


Hein.

[RAC_1]

Hein,

From following output,
21:35:23 device %busy avque r+w/s blks/s avwait avserv
21:35:24 c1t2d0 7.84 0.50 14 71 4.53 8.56
c2t2d0 8.82 0.50 12 63 4.33 8.12
c4t3d0 5.88 0.50 5 78 7.58 12.60
c4t4d0 4.90 0.50 4 63 7.46 14.34
c4t5d0 0.98 0.50 3 63 6.46 4.13
c5t5d0 0.98 0.50 2 47 6.04 7.54
c5t6d0 1.96 0.50 1 16 6.82 13.69
c12t0d4 1.96 20.96 77 700 9.19 0.78
c13t0d4 99.02 0.50 1 2 3.58 0.07
c12t1d4 1.96 0.50 7 135 4.67 2.37


I want,

21:35:23 device %busy avque r+w/s blks/s avwait avserv
21:35:24 c1t2d0 7.84 0.50 14 71 4.53 8.56
c13t0d4 99.02 0.50 1 2 3.58 0.07


Because, first two lines have timestamp and last line has %busy greater than 60 and avwait is more than avserv.

Your awk statement greps only first line.

[Hein van den Heuvel]

I did not have a system with sar handy when I replied just now. After a quick google I see there is a time stamp per list of records. What I now think you want is:

$ perl -nae'print if /avserv$/; $t=$1 if (/^(\d+:\d+:\d+)/); $F[-2]>=$F[-1]&&$F[-6]>=60 and print "$t $_"' x
03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c13t0d4 100.00 0.50 1 2 4.64 0.49

or

$ awk '/^[0-9][0-9]:/{t=$1} /avserv$/; /^c/{if ($2>60 && $(NF-1) > $NF)print t,$0}' x
03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c13t0d4 100.00 0.50 1 2 4.64 0.49

In both cases I load a variable 't' with the last timestamp seen where timestamp is recognized by soem decimals and colon at the begin of a line. I then print that last 't' seen along with any line matching the other criteria.

btw... Next time please attach a .txt file with the exact sample input data, and a sample of the desired output because the forum text mangles whitespace. It also prevents idle speculation, and re-replies about exactly what was requested.

Also, for your other question:
"Jan 5, 2006 10:11:12 GMT N/A: Question Author
----------------------------------------
where are those scripting wizards?? "

I think the answer is: "in bed in the USA"
Fortunately the UK, and the Netherland were present and awake.
:-)

Happy New Year,

Hein.

[RAC_1]

perl solution works, but not awk.

Can you check??

thank you for your help.

[Hein van den Heuvel]

>> Can you check??

Not without the attached file .txt with
- sample input.
- an indication how it failed.

As I showed, the awk worked for me... albeit on a linux box (my CD player :-).
The hpux awk may need some more parens or curly brackets....

Hein.

[RAC_1]

Attaching a file here. Need hp-ux awk solution.

[RAC_1]

Attaching a file here. Need hp-ux awk solution.

[Ermin Borovac]

You can try the following ...

awk '{if (NR==1 || (NF==8 && $3>60 && $(NF-1)>$NF) || (NF==7 && $2>60 && $(NF-1)>$NF)) print}' testfile

[RAC_1]

Ermin,

This works except that, I would like to have time stap at every record. (This timestamp would last encountered time stamp)

00:00:00 device %busy avque r+w/s blks/s avwait avserv
c12t0d4 0.58 134.53 17 165 22.34 1.44
c12t0d4 1.67 220.65 15 138 231.37 6.85
c12t0d4 0.98 104.79 21 184 25.27 1.36
c4t5d0 11.82 6.74 15 210 28.56 16.03


Should be as follows.
00:00:00 device %busy avque r+w/s blks/s avwait avserv
00:00:00 c12t0d4 0.58 134.53 17 165 22.34 1.44
00:00:00 c12t0d4 1.67 220.65 15 138 231.37 6.85

[Hein van den Heuvel]

As i suspected. My example failed because the actual data did not match the forum presenation which had stripped leading spaces. I tested for datalines looking for the a 'c' in the first position.

The variable field count can be solved by workign from teh right, as per my perl example.

The awk solution also needs to be improved by checking for more than 5 fields on a line.

Here a slightly different awk approach which I think will work for you: Once you replace the '^ ' by a '^' followed by 8 spaces.

awk '/^[0-9][0-9]:/{t=$1} /avserv$/; {if (NF>5 && $(NF-5)>60 && $(NF-1) > $NF){sub(/^ /,t,$0); print}}' xx

1) pick up timestamp into t, if you see one.
2) if you see a header line, do the default which is print.
3) if there are enough fields, and the 2 main test succeed then do 4
4) substitute any leading blanks if present by the last timestamp and print.

NOTE AGAIN: The /^ / is actually a series of 8 spaces!

Result:
22:24:18 device %busy avque r+w/s blks/s avwait avserv
22:24:19 c1t2d0 64.95 0.50 6 40 16.64 9.86
22:24:19 c4t5d0 65.45 0.50 76 1248 15.32 8.39

Now if you really do not want the duplicate timestamp then change to

awk '/^[0-9][0-9]:/{t=$1} /avserv$/; {if (NF>5 && $(NF-5)>60 && $(NF-1) > $NF){sub(/^ /,t,$0); print; t=" "}}' xx
22:24:18 device %busy avque r+w/s blks/s avwait avserv
22:24:19 c1t2d0 64.95 0.50 6 40 16.64 9.86
c4t5d0 65.45 0.50 76 1248 15.32 8.39


ONCE AGAIN: The quoted space is really 8 spaces, both in the sub regexpr and in the reset.

Hein.

[RAC_1]

I got it to working with following code. I would still like to get rid off second column.

awk '/^[0-9][0-9
]:/{t=$1} /avserv$/;{if (NR==1 || (NF==8 && $(NF-1)>250) || (NF==7 && $(NF-1)>250)) print t,$0}' xyz

[Hein van den Heuvel]

Hmm, That looks like a difference selection allt ogether.

anyway, re-read my prior reply. It solves the extra column.

also... please note the discrapency between the request in the attachment (no timestamp on second line) and later replies. That does not make it any easier to try to help.

cheers,
Hein.

[Doug O'Leary]

Hey;

Just to add my $.02... Attached is a perl script that will parse sar binary files and filter the output based on command line args:

-f ${file} # Sar file to process - defaults to today.
-d ${device} # Device to scan for - no default.
-p ${perc} # Percent busy - no default.
-q ${queue} # Average queue - no default.
-b ${blocks} # Blocks/sec - no default.
-w ${avgwio} # Average wio - defaults to 100ms
-s ${avserv} # Average svc time - no default.

HTH;

Doug

[Sandman!]

RAC,

Based on your input file here's a short awk construct that will do the job:

# awk '{if((NR==1) || $(NF-1)>$NF && $(NF-5)>60) print $0}'

FYI...pasted below is the input file I used
=============================================
22:24:18 device %busy avque r+w/s blks/s avwait avserv
22:24:19 c1t2d0 64.95 0.50 6 40 16.64 9.86
c2t2d0 2.97 0.50 4 32 7.02 8.87
c4t3d0 0.99 0.50 1 16 8.56 8.40
c4t4d0 0.99 0.50 1 16 8.58 8.08
c4t5d0 65.45 0.50 76 1248 15.32 8.39
c5t5d0 44.55 0.50 70 1141 4.97 6.52
=============================================

cheers!

[Ermin Borovac]

RAC, you should really reread Hein's last post especially

>>> NOTE AGAIN: The /^ / is actually a series of 8 spaces!

The last note applies to 'sub(/^ /,t,$0)' statement.

awk '/^[0-9][0-9]:/{t=$1} /avserv$/; {if (NF>5 && $(NF-5)>60 && $(NF-1) > $NF){sub(/^ /,t,$0); print}}' xx

In fact, because forum software discards whitespace, the last statement could be written as (note 'sub' statement changes).

awk '/^[0-9][0-9]:/{t=$1} /avserv$/; {if (NF>5 && $(NF-5)>60 && $(NF-1) > $NF){sub(/^[ ]{8}/,t,$0); print}}' xx