Generate list from file

[ust3]

I have a very large file that have many user name , the user name is separated by comma ( as below) , some of these names are duplicated ( like the below file , the name "tom" is duplicated ) , I would like to have a new name list which includes all user name but do NOT have the duplicated name , that mean only one "tom" in the new user name list , can advise what can i do ? thx.



$vi file_name
rebecca,tom, peter,ellen,edmond,leo,andy,jacky,ada,wendy,mandy,charles,anson,winkle,roger,jenny,tom,mary,venny,........


the new name list should be
rebecca,tom, peter,ellen,edmond,leo,andy,jacky,ada,wendy,mandy,charles,anson,winkle,roger,jenny,mary,venny,........

[Sandman! · Accepted solution]

Try the awk construct posted below:

# awk -F, '{for(i=1;i<=NF;++i){if(!x[$i]){x[$i]++;printf("%s,",$i)}}print ""}' file

[ust3]

Thx