Help on removing duplicate in 2 files

[Mike_305]

Hello,

I have 2 files and trying to remove duplicates from these two files.

How can I do this?

Appreciate your help and thanks in advance.

Thanks.

[James R. Ferguson]

Hi:

And what do you want to do with the duplicates --- retain them in one file or a new file or discard them from both?

What constitutes a duplicate --- a whole matching line or just a field within a line?

A bit better definition of the environment and the objective would be helpful.

Regards!

...JRF...

[Mike_305]

Hello James,

Absolutely, my bad. I am trying to compare two files that has the list disk like /dev/dsk/cXtYdZ.

Basically, I donâ t want to use the same disk that I have in LVMTAB and list I created.

Thanks in advance.

[Mike_305]

Hello,

If I use "grep" by it self I can see the duplicate and give me the output that is in both files but I need to figure out which is not duplicate disk and not in use by other VG's on the systems.

I am doing this and not helping.

for X in $(cat disk.list)
do
grep ${X} new.disk.list
done

An "-v" option give me somethong different.


Appreciate your help.

Thanks....

[James R. Ferguson]

HI (again):

You can use 'grep' and a file of the devices you want to sift:

Given that '/etc/lvmtab' is a binary file, we need to use 'strings'. You could do:

# strings /tmp/lvmtab|grep -v vg|grep -f /tmp/mydev

...and:

# strings /tmp/lvmtab|grep -v vg|grep -v -f /tmp/mydev

...where the /tmp/mydev file lists the devices one per line.

Regards!

...JRF...

[Mike_305]

Hello James,

I got the file ready by filtering the output from powermt command but now I am trying to insert /dev/dsk in front of it using "sed".

Once, I have that done I think I will be in place.

Thanks for your help.

[Dennis Handly · Accepted solution]

>I am doing this and not helping.
>for X in $(cat disk.list)

You need to use vector methods:
fgrep -v -f disk.list new.disk.list

This excludes all of the lines in disk.list from new.disk.list. (Other options are needed of some lines are substrings of other longer ones, add -x.)

>insert /dev/dsk in front of it using "sed".

sed 's:^:/dev/dsk:' file

[Mike_305]

Thanks for everyones help. Appreciate that very much.