topic Re: Format in Operating System - HP-UX

Format

SAM_24 — Tue, 10 Aug 2004 16:18:43 GMT

Hi,

How to delete unique lines based on particular field?

word1 TIME word2 tdce-512 word4
word1 TIME word2 tdce-512 word4
word1 TIME word2 tdce-512 word4
word1 TIME word2 tdce-513 word4
word1 TIME word2 tdce-517 word4

I want to delete duplicate lines based on fourth field.

Any help is appreciated.

Thanks.

Re: Format

Jeff_Traigle — Tue, 10 Aug 2004 16:21:33 GMT

Assuming the input is in a file named filename and you meant you want to keep one unique entry based on the fourth field, this should do it (-u for unique, -k4 for fourth field as key):

sort -u -k4 filename

Re: Format

Sundar_7 — Tue, 10 Aug 2004 16:22:40 GMT

Try this

sort -buk 4 < input-file > output-file

Re: Format

Hein van den Heuvel — Tue, 10 Aug 2004 17:08:50 GMT

sort will do the job, bot of course it will do the sort also.

If you need to keep the original order, yet must only retain the first of a duplicate then you can use a perl (or awk) script like:

perl -ne '$x=(split)[3];next if $seen{$x};$seen{$x}++;print' file.

Re: Format

Hein van den Heuvel — Wed, 11 Aug 2004 10:04:10 GMT

Hmmm, you assigned 4 points to each solutions presented, suggesting that you did evaluate them, but that they did NOT solve your problem.
Maybe you can clarify what is missing based on the results from the earlier replies? Maybe a more clear 'before' and 'after' picture, perhaps in a text attachment, is needed?

Regards,
Hein.

Re: Format

Rodney Hills — Wed, 11 Aug 2004 10:20:00 GMT

If you are looking to not use sort, then how about awk-

awk '{if (x != $4) { print $0}; x=$4}'

HTH

-- Rod Hills