Text filtering and formatting .....

[Karthik_2]

1.How do I remove duplicate lines in a
text file after sorting .
ex:
1111
2222
2222
3333
3333
I need to have only
1111
2222
3333
2.How do I sort lines in a file based on the
length of the line.
ex:
234
eqwr
45678
dfghhjk

cheers
Karthik ...

[Herve BRANGIER]

Hi

For you first question use "uniq" command
(man uniq). For example :

uniq foo.txt | sort

HTH

Herv?

[Andreas Voss · Accepted solution]

Hi,

with 'sort -u' you can do sort and uniq in one command.

For sorting length you can use this

cat | awk '{printf("%9d %s\n",length($0),$0);}' | sort -n | cut -c11-

Regards

[Herve BRANGIER]

Another way for the first question :

sort -u foo.txt

(-u : like uniq command)

For the second question try something like :

line_size.sh :

#!/bin/sh
for line in `sort -u $1`
do
echo `echo $line | wc -c` $line
done

And after use :

line_size.sh foo.txt | sort -k 1,1n | cut -f2 -d' '

HTH

Herv?

[Sridhar Bhaskarla]

This is the way I handle this.

Cat text1 > text
cat text1 >> text
sort text |uniq -d

Try this cheap method out.

-Sri