topic Re: Text filtering and formatting ..... in Operating System - HP-UX

Text filtering and formatting .....

Karthik_2 — Thu, 30 Aug 2001 05:39:43 GMT

1.How do I remove duplicate lines in a
text file after sorting .
ex:
1111
2222
2222
3333
3333
I need to have only
1111
2222
3333
2.How do I sort lines in a file based on the
length of the line.
ex:
234
eqwr
45678
dfghhjk

cheers
Karthik ...

Re: Text filtering and formatting .....

Herve BRANGIER — Thu, 30 Aug 2001 05:46:12 GMT

Hi

For you first question use "uniq" command
(man uniq). For example :

uniq foo.txt | sort

HTH

Herv?

Re: Text filtering and formatting .....

Andreas Voss — Thu, 30 Aug 2001 06:07:44 GMT

Hi,

with 'sort -u' you can do sort and uniq in one command.

For sorting length you can use this

cat | awk '{printf("%9d %s\n",length($0),$0);}' | sort -n | cut -c11-

Regards

Re: Text filtering and formatting .....

Herve BRANGIER — Thu, 30 Aug 2001 06:12:49 GMT

Another way for the first question :

sort -u foo.txt

(-u : like uniq command)

For the second question try something like :

line_size.sh :

#!/bin/sh
for line in `sort -u $1`
do
echo `echo $line | wc -c` $line
done

And after use :

line_size.sh foo.txt | sort -k 1,1n | cut -f2 -d' '

HTH

Herv?

Re: Text filtering and formatting .....

Sridhar Bhaskarla — Thu, 30 Aug 2001 14:12:45 GMT

This is the way I handle this.

Cat text1 > text
cat text1 >> text
sort text |uniq -d

Try this cheap method out.

-Sri