1833780 Members
2380 Online
110063 Solutions
New Discussion

Help in Vi

 
Qcheck
Super Advisor

Help in Vi

Hi All,

In a huge file how can I remove duplicate lines.

Where each line consists of only an username.

For instance, testuser is in three line randomnly in the file, I want to keep only one entry of testuser.

Please advise...

Thank you in advance!
4 REPLIES 4
James R. Ferguson
Acclaimed Contributor

Re: Help in Vi

Hi:

If the file is sorted, use 'uniq'. If the file isn't sorted, and you can sort it, use 'sort -u'. For example:

# uniq file file.out

# sort -u file > file.out

Regards!

...JRF...
Sandman!
Honored Contributor

Re: Help in Vi

Not sure how to do this in vi but here's an awk construct you could try...

# awk '{x[$0]++}END{for(i in x) print i}' infile

~cheers
Qcheck
Super Advisor

Re: Help in Vi

Thank you so much both James and Sandman!
You got your 10 points as always...
spex
Honored Contributor

Re: Help in Vi

Hello,

To remove duplicate lines while editing the file in vi, shell out to the OS:

:%!sort -u

PCS