topic Re: uniq sort and group in Operating System - Linux

uniq sort and group

Yashy — Sat, 20 Aug 2005 05:48:46 GMT

Hi,
I have a text file like below which has several hundred lines.

createuser ba026860 ac50003000
createuser na027483 ae50007009
createuser rax78895 af60003001
createuser tmxr8983 af50003001

The 3rd field in 3rd and 4th line is same. I want to sort it on the third field first, and print the output of the third field as uniq (grouping); like below (if the 3rd field has the same value; then group and print only one time as below:

createuser ba026860 ac50003000
createuser na027483 ae50007009
createuser rax78895 af60003001
createuser tmxr8983

Please advise.

Thanks.

Re: uniq sort and group

Victor Fridyev — Sat, 20 Aug 2005 09:33:39 GMT

Hi,

In order to sort a file by 3rd field you can use the command:
sort -k 3 filename
For the desired output you can use awk:
sort -k 3 filename| awk '{if($3!=prev){prev=$3;print}}'

HTH

Re: uniq sort and group

Leif Halvarsson_2 — Sat, 20 Aug 2005 09:34:44 GMT

Hi,
Sort the file on the third fild first.
Then, use the command "uniq -f 2 ".

The option "-f 2" ignores the first two filds when comparing the lines. Else the entire line is compared.

Re: uniq sort and group

Yashy — Sat, 20 Aug 2005 23:36:01 GMT

It does not work. I still get the output same as sort -k3

Re: uniq sort and group

Yashy — Sun, 21 Aug 2005 00:46:56 GMT

Sorry. The source file is like this

createuser ba026860 ac50003000
createuser na027483 ae50007009
createuser rax78895 af50003001
createuser tmxr8983 af50003001

The 3rd field in the last two lines are same.
So I need the result to come as below:

createuser ba026860 ac50003000
createuser na027483 ae50007009
createuser rax78895 af50003001
createuser tmxr8983

Thanks

Re: uniq sort and group

Hein van den Heuvel — Sun, 21 Aug 2005 01:22:32 GMT

Minor tweak on Victor's reply gets the desired output:

sort -k 3 your-file | awk '{x=($3==old)? "" : $3; old=$3; print $1,$2,x}'

For more complex options you may want to use a perl frame work and then tweak that.
For example:

-------------------- group.p ---------
%lines = <>;
sub sort_3_2 {
($a1,$a2,$a3) = split(/\s+/,$a);
($b1,$b2,$b3) = split(/\s+/,$b);
$a3.$a2 cmp $b3.$b2
}
foreach (sort sort_3_2 %lines) {
($a1,$a2,$a3) = split;
$x3 = ($a3 eq $old)? "" : $a3;
print "$a1 $a2 $x3\n";
$old = $a3;
}
--------------------------------

perl group.p your-file

hth,
Hein.

Re: uniq sort and group

Yashy — Sun, 21 Aug 2005 04:37:28 GMT

Hi Hein

The awk one does not work. The perl one works. But I cannot implement here as I do not know perl.

Can you please check the awk on.

This is the error I get form the awk.
awk: syntax error near line 1
awk: illegal statement near line 1

thanks.

Re: uniq sort and group

Hein van den Heuvel — Sun, 21 Aug 2005 09:54:26 GMT

Hmmm, the awk one really seems to work for me.
Did you re-type or cut & paste (and change the file name)

% uname -a
HP-UX hpgsp05c B.11.11 U 9000/800 690339353 unlimited-user license
% cat x
createuser ba026860 ac50003000
createuser na027483 ae50007009
createuser rax78895 af50003001
createuser tmxr8983 af50003001
createuser ba026861 ac50003000
createuser na027483 ae50007009
createuser na027485 ae50007009
createuser na027484 ae50007009
createuser rax78895 af50003001
createuser tmxr8983 af50003001
% sort -k 3 x | awk '{x=($3==old)? "" : $3; old=$3; print $1,$2,x}'
createuser ba026860 ac50003000
createuser ba026861
createuser na027483 ae50007009
createuser na027483
createuser na027484
createuser na027485
createuser rax78895 af50003001
createuser rax78895
createuser tmxr8983
createuser tmxr8983

fwiw,
Hein.

Re: uniq sort and group

Muthukumar_5 — Mon, 22 Aug 2005 00:31:30 GMT

You can do it as,

sort -k 3 | awk '{ if ( prev != $3 ) { prev=$3;print; } else { print $1" "$2 }}'

hth.

Re: uniq sort and group

Yashy — Mon, 22 Aug 2005 23:51:26 GMT

Thanks Muthu. It is exactly what I was looking for.

Yashy.