Scripting question

[John McDen]

I have few files which is generated every week it has few duplicate entries in it.

Is it possible to do something like this

input file :

EXPZ_84940
KJAD_02938
KJAD_02938
NJSK_77490
....

output file:
EXPZ_84940
KJAD_02938 KJAD_02938
NJSK_77490
....

so that all the matching ones are beside each other.

Thanks for your time and help.

[Paula J Frazer-Campbell]

Hi

Try

Cat file | sort > newfile


Paula

[John McDen]

Hi Paula,

That will only sort the files I have the files sorted but I need it to place the duplicate entry next to each other in the same line, like in the output.

Thanks for your help.

[Andreas Voss]

Hi,

uniq will do that for you:

uniq file > newfile

Regards

[Andreas Voss]

Hi again,

sorry i haven't read your question exactly.
Here my little awk script for your job:

awk 'BEGIN{p=""}{
if(p=="")
p=$1
else
{
printf p;
if(p==$1)
printf " ";
else
printf "\n";
p=$1;
}
}END{print $1}' file >newfile

Regards

[John McDen]

Hello Andreas,

Thanks for your help and efforts but I get this error ..may be I am doing some thing wrong.

$ awk -f spl.awk tpp.txt
syntax error The source line is 1.
The error context is
awk >>> ' <<<
awk: Quitting
The source line is 1.
$

[Darrell Allen · Accepted solution]

Hi John,

Here's a way to do so in a script:

while read line
do
echo "$line\c"
prev=$line
while read line
do
if [ $prev = $line ]
then
echo " $line\c"
else
echo "\n$line\c"
fi
prev=$line
done
done otfile
echo >>otfile

This only works if the duplicated lines are consecutive in the file. You could sort the file first to ensure duplicates follow each other.

Darrell

[John McDen]

Hi Darrell Allen

I wish I could give you more then 10 points .. it works really great...

Thanks a Million

[Darrell Allen]

You're welcome (and N/A for this please).

Darrell