topic Re: awk/perl help. in Operating System - HP-UX

awk/perl help.

RAC_1 — Thu, 05 Jan 2006 04:39:52 GMT

Putting it here to get more responses. I need perl/awk help from you guys. simple, but proving to be difficult for me.

HP-UX xxxyyy B.11.11 U 9000/800 01/05/06

03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c1t2d0 11.88 0.50 21 97 4.35 10.33
c2t2d0 10.89 0.50 19 89 4.50 7.35
c4t3d0 0.99 0.50 1 16 2.89 16.59
c4t4d0 0.99 0.50 1 16 2.90 12.09
c4t5d0 0.99 0.50 1 16 1.53 9.00
c5t5d0 0.99 0.50 1 16 1.54 11.23
c12t0d3 0.99 0.50 7 453 6.54 1.62
c13t0d4 100.00 0.50 1 2 4.64 0.49
c12t1d4 0.99 0.50 7 137 3.20 1.13
c12t0d0 1.98 0.50 33 507 5.08 0.60

I have following output (from sar -f) I need to get output based on following criteria.

Print a record if (last second last column (avwait) is more than aserver (last column))
further, %busy is more than 60.

I tried following, but not getting what I want.
cat "xxx" |Average"|awk '{if (($1~":") && ($3>60) && $((NF-1))>$NF) print $0;else if (($1!~":") && ($2>60) && $((NF-1))>$NF) print $0;else print "" ;}'

I should get only following lines.

03:32:24 device %busy avque r+w/s blks/s avwait avserv
c13t0d4 100.00 0.50 1 2 4.64 0.49

Re: awk/perl help.

RAC_1 — Thu, 05 Jan 2006 05:11:12 GMT

where are those scripting wizards??

Re: awk/perl help.

Peter Godron — Thu, 05 Jan 2006 06:14:35 GMT

RAC,
is the output not correct?
The only record with %busy > 60 is record 8 (c13t0d4)
I copied the data into file xxx ,copied your statement into a sh and changed
the beginning to
cat xxx | awk ...
and got
c13t0d4 100.00 0.50 1 2 4.64 0.49

What are you getting as output ?

Re: awk/perl help.

H.Merijn Brand (procura — Thu, 05 Jan 2006 06:19:16 GMT

Simple :) :)

lt09:/home/merijn 103 > cat xx.txt
03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c1t2d0 11.88 0.50 21 97 4.35 10.33
c2t2d0 10.89 0.50 19 89 4.50 7.35
c4t3d0 0.99 0.50 1 16 2.89 16.59
c4t4d0 0.99 0.50 1 16 2.90 12.09
c4t5d0 0.99 0.50 1 16 1.53 9.00
c5t5d0 0.99 0.50 1 16 1.54 11.23
c12t0d3 0.99 0.50 7 453 6.54 1.62
c13t0d4 100.00 0.50 1 2 4.64 0.49
c12t1d4 0.99 0.50 7 137 3.20 1.13
lt09:/home/merijn 104 > perl -nae'$F[-2]>=$F[-1]&&$F[2]>=.6 and print' xx.txt
lt09:/home/merijn 105 > perl -nae'$F[-2]>=$F[-1]&&$F[2]>=.5 and print' xx.txt
c12t0d3 0.99 0.50 7 453 6.54 1.62
c13t0d4 100.00 0.50 1 2 4.64 0.49
c12t1d4 0.99 0.50 7 137 3.20 1.13
lt09:/home/merijn 106 >

Enjoy, Have FUN! H.Merijn

Re: awk/perl help.

H.Merijn Brand (procura — Thu, 05 Jan 2006 06:46:04 GMT

Small change, as I took the headers of the first line to the meaning of the columns

lt09:/home/merijn 102 > perl -nae'$F[-2]>=$F[-1]&&$F[1]>=60 and print' xx.txt
c13t0d4 100.00 0.50 1 2 4.64 0.49
lt09:/home/merijn 103 >

Enjoy, Have FUN! H.Merijn

Re: awk/perl help.

RAC_1 — Thu, 05 Jan 2006 22:01:09 GMT

Peter,

I am not getting correct output with my awk command. I also need to get following lines.

00:00:00 device %busy avque r+w/s blks/s avwait avserv

and first line in output.

these two lines have timestamp, which is required to associate that data with time when it happened.

Procura,

How to include above requirement in perl statement?

Re: awk/perl help.

Hein van den Heuvel — Thu, 05 Jan 2006 22:29:47 GMT

perl -nae'/avserv/ || $F[-2]>=$F[-1]&&$F[1]>=60 and print' x

or

awk '/avserv$/; /^c/{if ($2>60 && $(NF-1) > $NF)print}' x

but...

What is that "03:32:25" on the line with "c1t2d0"?
Is that for real or cut&paste thing?

Hein.

Re: awk/perl help.

RAC_1 — Thu, 05 Jan 2006 22:37:33 GMT

Hein,

From following output,
21:35:23 device %busy avque r+w/s blks/s avwait avserv
21:35:24 c1t2d0 7.84 0.50 14 71 4.53 8.56
c2t2d0 8.82 0.50 12 63 4.33 8.12
c4t3d0 5.88 0.50 5 78 7.58 12.60
c4t4d0 4.90 0.50 4 63 7.46 14.34
c4t5d0 0.98 0.50 3 63 6.46 4.13
c5t5d0 0.98 0.50 2 47 6.04 7.54
c5t6d0 1.96 0.50 1 16 6.82 13.69
c12t0d4 1.96 20.96 77 700 9.19 0.78
c13t0d4 99.02 0.50 1 2 3.58 0.07
c12t1d4 1.96 0.50 7 135 4.67 2.37

I want,

21:35:23 device %busy avque r+w/s blks/s avwait avserv
21:35:24 c1t2d0 7.84 0.50 14 71 4.53 8.56
c13t0d4 99.02 0.50 1 2 3.58 0.07

Because, first two lines have timestamp and last line has %busy greater than 60 and avwait is more than avserv.

Your awk statement greps only first line.

Re: awk/perl help.

Hein van den Heuvel — Thu, 05 Jan 2006 22:58:27 GMT

I did not have a system with sar handy when I replied just now. After a quick google I see there is a time stamp per list of records. What I now think you want is:

$ perl -nae'print if /avserv$/; $t=$1 if (/^(\d+:\d+:\d+)/); $F[-2]>=$F[-1]&&$F[-6]>=60 and print "$t $_"' x
03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c13t0d4 100.00 0.50 1 2 4.64 0.49

or

$ awk '/^[0-9][0-9]:/{t=$1} /avserv$/; /^c/{if ($2>60 && $(NF-1) > $NF)print t,$0}' x
03:32:24 device %busy avque r+w/s blks/s avwait avserv
03:32:25 c13t0d4 100.00 0.50 1 2 4.64 0.49

In both cases I load a variable 't' with the last timestamp seen where timestamp is recognized by soem decimals and colon at the begin of a line. I then print that last 't' seen along with any line matching the other criteria.

btw... Next time please attach a .txt file with the exact sample input data, and a sample of the desired output because the forum text mangles whitespace. It also prevents idle speculation, and re-replies about exactly what was requested.

Also, for your other question:
"Jan 5, 2006 10:11:12 GMT N/A: Question Author
----------------------------------------
where are those scripting wizards?? "

I think the answer is: "in bed in the USA"
Fortunately the UK, and the Netherland were present and awake.
:-)

Happy New Year,

Hein.

Re: awk/perl help.

RAC_1 — Thu, 05 Jan 2006 23:03:11 GMT

perl solution works, but not awk.

Can you check??

thank you for your help.

Re: awk/perl help.

Hein van den Heuvel — Thu, 05 Jan 2006 23:19:39 GMT

>> Can you check??

Not without the attached file .txt with
- sample input.
- an indication how it failed.

As I showed, the awk worked for me... albeit on a linux box (my CD player :-).
The hpux awk may need some more parens or curly brackets....

Hein.

Re: awk/perl help.

RAC_1 — Thu, 05 Jan 2006 23:28:15 GMT

Attaching a file here. Need hp-ux awk solution.

Re: awk/perl help.

RAC_1 — Thu, 05 Jan 2006 23:40:08 GMT

Attaching a file here. Need hp-ux awk solution.

Re: awk/perl help.

Ermin Borovac — Fri, 06 Jan 2006 00:44:54 GMT

You can try the following ...

awk '{if (NR==1 || (NF==8 && $3>60 && $(NF-1)>$NF) || (NF==7 && $2>60 && $(NF-1)>$NF)) print}' testfile

Re: awk/perl help.

RAC_1 — Fri, 06 Jan 2006 00:55:58 GMT

Ermin,

This works except that, I would like to have time stap at every record. (This timestamp would last encountered time stamp)

00:00:00 device %busy avque r+w/s blks/s avwait avserv
c12t0d4 0.58 134.53 17 165 22.34 1.44
c12t0d4 1.67 220.65 15 138 231.37 6.85
c12t0d4 0.98 104.79 21 184 25.27 1.36
c4t5d0 11.82 6.74 15 210 28.56 16.03

Should be as follows.
00:00:00 device %busy avque r+w/s blks/s avwait avserv
00:00:00 c12t0d4 0.58 134.53 17 165 22.34 1.44
00:00:00 c12t0d4 1.67 220.65 15 138 231.37 6.85

Re: awk/perl help.

Hein van den Heuvel — Fri, 06 Jan 2006 01:26:44 GMT

As i suspected. My example failed because the actual data did not match the forum presenation which had stripped leading spaces. I tested for datalines looking for the a 'c' in the first position.

The variable field count can be solved by workign from teh right, as per my perl example.

The awk solution also needs to be improved by checking for more than 5 fields on a line.

Here a slightly different awk approach which I think will work for you: Once you replace the '^ ' by a '^' followed by 8 spaces.

awk '/^[0-9][0-9]:/{t=$1} /avserv$/; {if (NF>5 && $(NF-5)>60 && $(NF-1) > $NF){sub(/^ /,t,$0); print}}' xx

1) pick up timestamp into t, if you see one.
2) if you see a header line, do the default which is print.
3) if there are enough fields, and the 2 main test succeed then do 4
4) substitute any leading blanks if present by the last timestamp and print.

NOTE AGAIN: The /^ / is actually a series of 8 spaces!

Result:
22:24:18 device %busy avque r+w/s blks/s avwait avserv
22:24:19 c1t2d0 64.95 0.50 6 40 16.64 9.86
22:24:19 c4t5d0 65.45 0.50 76 1248 15.32 8.39

Now if you really do not want the duplicate timestamp then change to

awk '/^[0-9][0-9]:/{t=$1} /avserv$/; {if (NF>5 && $(NF-5)>60 && $(NF-1) > $NF){sub(/^ /,t,$0); print; t=" "}}' xx
22:24:18 device %busy avque r+w/s blks/s avwait avserv
22:24:19 c1t2d0 64.95 0.50 6 40 16.64 9.86
c4t5d0 65.45 0.50 76 1248 15.32 8.39

ONCE AGAIN: The quoted space is really 8 spaces, both in the sub regexpr and in the reset.

Hein.

Re: awk/perl help.

RAC_1 — Fri, 06 Jan 2006 01:52:33 GMT

I got it to working with following code. I would still like to get rid off second column.

awk '/^[0-9][0-9
]:/{t=$1} /avserv$/;{if (NR==1 || (NF==8 && $(NF-1)>250) || (NF==7 && $(NF-1)>250)) print t,$0}' xyz

Re: awk/perl help.

Hein van den Heuvel — Fri, 06 Jan 2006 07:42:45 GMT

Hmm, That looks like a difference selection allt ogether.

anyway, re-read my prior reply. It solves the extra column.

also... please note the discrapency between the request in the attachment (no timestamp on second line) and later replies. That does not make it any easier to try to help.

cheers,
Hein.

Re: awk/perl help.

Doug O'Leary — Fri, 06 Jan 2006 08:34:56 GMT

Hey;

Just to add my $.02... Attached is a perl script that will parse sar binary files and filter the output based on command line args:

-f ${file} # Sar file to process - defaults to today.
-d ${device} # Device to scan for - no default.
-p ${perc} # Percent busy - no default.
-q ${queue} # Average queue - no default.
-b ${blocks} # Blocks/sec - no default.
-w ${avgwio} # Average wio - defaults to 100ms
-s ${avserv} # Average svc time - no default.

HTH;

Doug

Re: awk/perl help.

Sandman! — Fri, 06 Jan 2006 13:26:59 GMT

RAC,

Based on your input file here's a short awk construct that will do the job:

# awk '{if((NR==1) || $(NF-1)>$NF && $(NF-5)>60) print $0}'

FYI...pasted below is the input file I used
=============================================
22:24:18 device %busy avque r+w/s blks/s avwait avserv
22:24:19 c1t2d0 64.95 0.50 6 40 16.64 9.86
c2t2d0 2.97 0.50 4 32 7.02 8.87
c4t3d0 0.99 0.50 1 16 8.56 8.40
c4t4d0 0.99 0.50 1 16 8.58 8.08
c4t5d0 65.45 0.50 76 1248 15.32 8.39
c5t5d0 44.55 0.50 70 1141 4.97 6.52
=============================================

cheers!