1844208 Members
2002 Online
110229 Solutions
New Discussion

Re: delete duplicates ..

 
SOLVED
Go to solution
someone_4
Honored Contributor

delete duplicates ..

Hey everyone ..
quick on here ..
I have a list of numbers like

1
1
1
2
2
3
3
3
4
4
4
4

I need to delete the duplicates and just endup with one of each

1
2
3
4

Thanks
Richard
11 REPLIES 11
Eric Ladner
Trusted Contributor
Solution

Re: delete duplicates ..

If it's in a file, say unsorted_file:

sort -u unsorted_file > sorted_file

Or, in a vi session, you can do this:

:1,$!sort -u
Joseph C. Denman
Honored Contributor

Re: delete duplicates ..

Hi Richard.

man uniq

Hope this helps.

...jcd...
If I had only read the instructions first??
harry d brown jr
Honored Contributor

Re: delete duplicates ..

Eric,

that vi sort is great! I learned something new, thing my boss will let me go home for the day?

live free or die
harry
Live Free or Die
someone_4
Honored Contributor

Re: delete duplicates ..

I agree that was good.

Richard
Eric Ladner
Trusted Contributor

Re: delete duplicates ..

Heh..

You can run ANY command like that, too..

Want to put a directory listing in the file you are editing? Here's how:

:.!ls -l

(make sure you are on a clean line, or it get's whacked).

In general, you send a range to the command after the bang, then any ouput get's inserted back into your current buffer.
someone_4
Honored Contributor

Re: delete duplicates ..

I like that ! Good Stuff !!

Richard
Anthony khan
Frequent Advisor

Re: delete duplicates ..

Eric,

that's sweet
H.Merijn Brand (procura
Honored Contributor

Re: delete duplicates ..

Are you addicted to the command mode of vi?

in window mode (editing, not input)

replace the current line with the uname output

!!uname -a

Sort the contents of a file immediately after opening

!Gsort -u

No problem to put pipes in there too.
Take out the dups (including symlinks) and nonexisting directories in a directory listing:

!Gperl -ne '-d&&$P{$_}++==0&&push@P,$_;}END{print join":",@P;'

If the command is to be pasted from a nice mail someone sent you, open a new line and paste that text into that line, then delete the complete line into a named buffer (say m) and execute the command using "@m"

So if the current display has a line containing

:r /tmp/swagent.log

and you delete the line using

"mdd

you can now go to the line after wich you want the contents of /tmp/swagent.log inserted and type

@m

More tips like these can be found on a well written place like http://www.networkcomputing.com/unixworld/tutorial/009/009.html
Enjoy, Have FUN! H.Merijn
Bill Hassell
Honored Contributor

Re: delete duplicates ..

While we're at it, one of the best places to find vi (actually, ed command lines) is the "Ultimate Guide to vi", a virtual requirement for any Unix sysadmin. One of the examples is a spell-checker for vi:

G
:r! spell

In fact, you can assing this to an unused vi command in your .exrc file. The example uses S since this isn't defined in vi:

map S G:w!^M:r!spell %^M

The ^M are literal carriage returns and must be added using the CTRL-V option flag in vi.

For word processing, you can use the adjust command to format paragraphs using word wrap:

!)exec adjust -m60

This command starts on the current line and works to the end of the paragraph (next blank line), adjusting the righthand margin to no more than 60 characters but breaking the lines on a word boundary. adjust is fairly coll in that it pays attention to indents.


Bill Hassell, sysadmin
steven Burgess_2
Honored Contributor

Re: delete duplicates ..

To spell-check without leaving a vi session:

:w !spell -b

take your time and think things through
someone_4
Honored Contributor

Re: delete duplicates ..

Thanks Mr. Bill for the extra info .. you always come up with some great additional stuff. Ohhh and I do have the vi book !!


Richard