combine records in file with awk?

[Stuart Abramson]

I have a file that looks like this:

270095000 1527 REG 8838720 vgeb_rdp0_q c100t5d2
270095000 1527 REG 8838720 vgeb_rdp0_q c91t5d2
270098000 1527 M(8) 70709760 Avail c100t5d5
270098000 1527 M(8) 70709760 Avail c91t5d5
2700A5000 1527 M(8) 70709760 Avail c100t2d2
2700A5000 1527 M(8) 70709760 Avail c91t2d2
2700A7000 1527 M(8) 70709760 Avail c100t1d6
2700A7000 1527 M(8) 70709760 Avail c91t1d6
2700C5000 1527 REG 8838720 vgeb_rdp0_q c101t10d7
2700C5000 1527 REG 8838720 vgeb_rdp0_q c92t10d7

We combine a daily "syminq" with "/etc/lvmtab" to figure out which PVs/HVs are in what volume groups.

I want to combine records where field 1 is the same, in such a way that I get the complete record 1, and the LAST FIELD from subsequent records with the same field 1:

270095000 1527 REG 8838720 vgeb_rdp0_q c100t5d2 c91t5d2

Can you do that with awk? (We already do it with "ksh" but it's clumsy.)

[Jean-Luc Oudart · Accepted solution]

Hi,

sort | awk '
BEGIN{fld1=$1;printf("%s",$0); getline;}
{if($1==fld1) { printf("%s",$6); }
else { fld1=$1; printf("\n%s",$0); }
}
END{printf("\n")}'

Regards
Jean-Luc

[Tim Nelson]

Check this script out.
Gathers all disk information, reports vg, lv, mounts, size, and free extents

Output in both formatted text and csv

[Hein van den Heuvel]

With your data lines in a file 'x', the following seems to work:

awk '{if ($1==old){line = line $NF} else { if (old){print line}; line=$0; old=$1 }}END{print line}' x

270095000 1527 REG 8838720 vgeb_rdp0_q c100t5d2 c91t5d2
270098000 1527 M(8) 70709760 Avail c100t5d5 c91t5d5
2700A5000 1527 M(8) 70709760 Avail c100t2d2 c91t2d2
2700A7000 1527 M(8) 70709760 Avail c100t1d6 c91t1d6
2700C5000 1527 REG 8838720 vgeb_rdp0_q c101t10d7 c92t10d7

Build up a print line in variable 'line'
If the current input starts out the same, then add the current last field to 'line'.
If not the same, print line (unless first), and reset to current input.
At end print print the final line assembled.

Hein.

[Volker Borowski]

Hi,

did you consider the join command ?
Should do exactly what you like to do.

Volker

[James R. Ferguson]

Hi Stuart:

You can easily do this in perl along these lines:

#!/usr/bin/perl -an
if ($F[0] eq $prev) {
push( @list, $F[5] )
} else {
print "$prev @list\n" if (@list);
$prev = $F[0];
@list = ();
push( @list, $F[5] );
}
END { print "$prev @list\n" }

Regards!

...JRF...

[Sandman!]

I second Hein's awk solution...quite elegantly and artfully done.

regards!

[Stuart Abramson]

All good answers. Thanks very much.

I forgot something. Look at this case:

2700A7000 1527 M(8) 70709760 Avail c100t1d6
2700A7000 1527 M(8) 70709760 Avail c91t1d6
2700C5000 1527 REG 8838720 Avail c101t10d7
2700C5000 1527 REG 8838720 vgeb_rd c92t10d7
2700C5000 1527 REG 8838720 Avail c95t10d7

Volume 00C5 is presented 3 times with three device file entries, but only one of them is added to the VG. So, I want the combined record to look like this:

2700C5000 1527 REG 8838720 vgeb_rd c92t10d7 c101t10d7 c95t10d7

I need some logic that says "if any of the X volumes with the same Serial Number are in a Volume Group, use that VG name, not the Avail name.."

[Hein van den Heuvel]

easy... in my example, just try replace 'Avail' with the current $5 field.
The will only do something once.
In an awk script:


{if ($1==old) {
line = line $NF
sub (/Avail/,$5,line);
} else {
if (old) {
print line;
}
line=$0;
old=$1;
}
}
END { print line }

Hein.

[Sandman!]

Stuart,

Here's an awk construct that outputs what you're looking for. It concatenates multiple instances of the first field in the input file with a single stream in the output file.

# awk '
> {
> if(NR==1) {
> line=$0
> prev=$1
> }
> else
> if($1==prev)
> line=line" "$NF
> else {
> print line
> line=$0
> prev=$1
> }
> }
>
> END {print line}
> ' input_file

[Stuart Abramson]

Thanks all, especially Hein.

Stuart