Script Help

[Sivasingam Santhakumar]

Hi,

I have the collect out put like this:

#### RECORD 1 (1083645776:78) (Tue May 4 14:42:56 2004) ####

# DISK Statistics
#DSK NAME B/T/L R/S RKB/S W/S WKB/S AVS AVW ACTQ WTQ %BSY
0 dsk0 0/0/0 0 1 4 39 6.04 0.00 0.03 0.00 1.95
1 dsk1 0/1/0 0 1 4 39 6.07 0.00 0.03 0.00 1.95
2 dsk2 0/2/0 14 166 10 80 6.10 0.00 0.15 0.00 14.44
3 dsk3 0/3/0 14 180 10 80 6.17 0.00 0.15 0.00 14.60
4 dsk4 0/4/0 0 0 0 0 0.00 0.00 0.00 0.00 0.00
5 cdrom0 1/0/0 0 0 0 0 0.00 0.00 0.00 0.00 0.00

#### RECORD 2 (1083646376:0) (Tue May 4 14:52:56 2004) ####

# DISK Statistics
#DSK NAME B/T/L R/S RKB/S W/S WKB/S AVS AVW ACTQ WTQ %BSY
0 dsk0 0/0/0 0 2 6 56 5.47 0.00 0.04 0.00 2.77
1 dsk1 0/1/0 0 0 6 56 5.47 0.00 0.04 0.00 2.81
2 dsk2 0/2/0 1 29 2 22 6.11 0.00 0.03 0.00 2.86
3 dsk3 0/3/0 2 32 2 22 6.07 0.00 0.03 0.00 2.83
4 dsk4 0/4/0 0 0 0 0 0.00 0.00 0.00 0.00 0.00
5 cdrom0 1/0/0 0 0 0 0 0.00 0.00 0.00 0.00 0.00

And I am getting an out put like this after passing it through SED and AWK on Tru64 machine

dsk0 | 1 | 39 | 1.95
dsk1 | 1 | 39 | 1.95
dsk2 | 166 | 80 | 14.44
dsk3 | 180 | 80 | 14.60
dsk4 | 0 | 0 | 0.00
dsk0 | 2 | 56 | 2.77
dsk1 | 0 | 56 | 2.81
dsk2 | 29 | 22 | 2.86
dsk3 | 32 | 22 | 2.83
dsk4 | 0 | 0 | 0.00

I am filtering out only fields 2, 5,7 and 12. I want to append the date and time (from the header record) to the end. Sample:
dsk0 | 1 | 39 | 1.95 | 04/05/2004 14:42

Is it possible?

Thanks Gurus

[Abdul Rahiman]

Hi Siva,

I wanted to do the same thing that you are looking for a while back and I was too lazy to do that then.. And I think I finally have come up with a workable script using some awk features .. Assuming that that the data records that you want to parse repeat i a fixed sequence...I assumed that each "RECORD" line is followed by 4 lines and then there 5 lines with dsk string on it.
This script will work as long as the pattern repeats in that fixed sequence. As for the date formatting, you may want to do some more formatting ..

ifile=field.txt
for i in 1 2
do
a=`grep -n "RECORD $i" $ifile | awk -F: '{print $1}'`
d=`grep -n "RECORD $i" $ifile |cut -c34-56`
x=`expr $a + 4`;y=`expr $a + 9`
grep -n dsk $ifile | awk -F ":" -v x=$x -v y=$y -v d="$d" '$1 >= x && $1 <= y {print}' | awk '{print $2"|"$5"|"$7"|" d}' | awk -v d="$d" '{print $1 d}'
done

o/p is

estprd_rahiman> ./collect.sh
dsk0|1|39|Tue May 4 14:42:56 2004
dsk1|1|39|Tue May 4 14:42:56 2004
dsk2|166|80|Tue May 4 14:42:56 2004
dsk3|180|80|Tue May 4 14:42:56 2004
dsk4|0|0|Tue May 4 14:42:56 2004
dsk0|2|56|Tue May 4 14:52:56 2004
dsk1|0|56|Tue May 4 14:52:56 2004
dsk2|29|22|Tue May 4 14:52:56 2004
dsk3|32|22|Tue May 4 14:52:56 2004
dsk4|0|0|Tue May 4 14:52:56 2004

Let me know if it worked for you .. script is attached here,

regds,
Abdul Rahiman.

[Mark Grant]

This is a perl version that doesn't really care how many entries there are or that the pattern repeats in the same way. I got a bit bored of the date formatting also so you might want to change that a bit.

#!/usr/bin/perl

open INPUT,"datafile";
$CURRENTDATE="";

while(){
s/.*RECORD.+\(\w+ +\w+ \d+ (.*)\).*/$CURRENTDATE=$1;print "" }/e;
next if /^[#,\n].*/ ;
($o,$tw,$th,$f)=($_=~/\S+ (\S+) \S+ \S+ (\S+) \S+ (\S+) \S+ \S+ \S+ \S+ (\S+).*/);
next if $o eq "";
print "$o|$tw|$th|$f|$CURRENTDATE\n";
}

[Sivasingam Santhakumar]

Mark,

Thanks for that perl script. My actual input file is Tab delimited and when put it on the web its changed to space. Sample input is attached. Could you show me steps for this input please.

Abdul,
Thanks for yours too. But have about 2000 records and perfer the perl one.

Thanks.
Siva

[Mark Grant]

The line

($o,$tw,$th,$f)=($_=~/\S+ (\S+) \S+ \S+ (\S+) \S+ (\S+) \S+ \S+ \S+ \S+ (\S+).*/);

Is probably a bit dumb anyway. You should be able to replace this with something like

($w,$o,$w,$w,$tw,$w,$th,$w,$w,$w,$w,$f)=split /\s/;

Which should work with tabs too. I'll check this just as soon as I get near a perl interpreter.

[Sivasingam Santhakumar]

Mark,

Still it doesn't work. I am using multiple awks
to process the file ( as I have to produce some reports for customer). I will need your perl for future months, so I can automate the reporting part.

Thanks heaps.
Siva

[Abdul Rahiman]

Siva,

>>I will need your perl for future months, >>so I can automate the reporting part.

>>>Thanks heaps.

Try to assign some points to the people who are taking time to respond to your questions .. it means a lot to us ..that would also help you get quicker and better responses in the future..

regds,
Abdul.

[Hein van den Heuvel · Accepted solution]

Siva, when you write "Mark, ...Still it doesn't work." what does that mean? No output, or just not exactly right.

If you need further help on this, then I suggest you reply with a text attachement with the exact example (spaces, tabs, end-of-lines) as needed, and perhaps also a few sample desired output lien examples.

Here is some perl that SEEMS to work:

perl -n x.pl x
dsk0 1 39 1.95 (Tue May 4 14:42:56 2004)
dsk1 1 39 1.95 (Tue May 4 14:42:56 2004)
dsk2 166 80 14.44 (Tue May 4 14:42:56 2004)
dsk3 180 80 14.60 (Tue May 4 14:42:56 2004)
dsk4 0 0 0.00 (Tue May 4 14:42:56 2004)
dsk0 2 56 2.77 (Tue May 4 14:52:56 2004)
dsk1 0 56 2.81 (Tue May 4 14:52:56 2004)
dsk2 29 22 2.86 (Tue May 4 14:52:56 2004)
dsk3 32 22 2.83 (Tue May 4 14:52:56 2004)
dsk4 0 0 0.00 (Tue May 4 14:52:56 2004)

x is a file containing the text from your note.
x.pl contains:
$time = $1 if (/^####.*(\([MTWFS].*\))/);
if (/^\s*\d+\s+dsk/) {
($w,$o,$w,$w,$tw,$w,$th,$w,$w,$w,$w,$f)=split;
print "$o $tw $th $f $time\n";
}

The first line looks for lines starting with #### and an opening paren immediatly followed by a first letter of a day. If it sees that is remembers (in $1) that paren and everything up to and including the closing parenthesis.
The next line looks for lines starting with optional whitespace, a number, whitespace and the letters 'dsk'.
If found use a default split (based on whitespace in $_) and print selected fields.

hth,
Hein.

[Mark Grant]

I just tried the script and apart from a missing "{" before "$CURRENTDATE=$1;print "" }" it sort of works. Can you let me know any error.

You might need to adjust Heins' script to allow for the "cdrom" entries.

[Sivasingam Santhakumar]

Thanks Hein for the solution with expected answer.

Thanks Mark for your time and effort. I still don't get the correct result (like Hein's). My out put is:
}||||00:03:26 2004
}||||00:13:26 2004
}||||00:23:26 2004
}||||00:33:26 2004
}||||00:43:26 2004
}||||00:53:26 2004
}||||01:03:26 2004
Like this. I am doing something wrong here.
my line is :
s/.*RECORD.+\(\w+ +\w+ \d+ (.*)\).*/$CURRENTDATE=$1;print "" }/e;

Thanks goes to Abdul also.

[Mark_399]

Hi

I use cfilt to extract data from collect output as follows:

collect -p 2004:06:23:10:30:00,2004:06:23:10:35:00|cfilt dis:name=dsk22,dsk23:rk
b/s+wkb/s:r/s+w/s|awk '{print $1","$3","$4","$5","$6}'|awk -F: '{print $3"/"$2"/"$1,$4":"$5":"$6}'

Bit long winded, but gives me output as follows which shows total I/O and transactions/sec for disks dsk22&23 with the date:
23/06/2004 10:30:10,13090,849,0,0
23/06/2004 10:30:20,12990,888,0,0
23/06/2004 10:30:30,14004,930,0,0

Cheers
Mark

[Sivasingam Santhakumar]

Thanks Mark,

Is it comes with 5.1, because I can see only collect on my system neither collgui or cfilt

Siva

[Mark_399]

Hi

I have attached cfilt (perl script).

Thanks
Mark