- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Help on removing duplicate in 2 files
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 02:03 PM
тАО05-13-2009 02:03 PM
I have 2 files and trying to remove duplicates from these two files.
How can I do this?
Appreciate your help and thanks in advance.
Thanks.
Solved! Go to Solution.
- Tags:
- grep
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 02:21 PM
тАО05-13-2009 02:21 PM
Re: Help on removing duplicate in 2 files
And what do you want to do with the duplicates --- retain them in one file or a new file or discard them from both?
What constitutes a duplicate --- a whole matching line or just a field within a line?
A bit better definition of the environment and the objective would be helpful.
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 02:46 PM
тАО05-13-2009 02:46 PM
Re: Help on removing duplicate in 2 files
Absolutely, my bad. I am trying to compare two files that has the list disk like /dev/dsk/cXtYdZ.
Basically, I don├в t want to use the same disk that I have in LVMTAB and list I created.
Thanks in advance.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 03:07 PM
тАО05-13-2009 03:07 PM
Re: Help on removing duplicate in 2 files
If I use "grep" by it self I can see the duplicate and give me the output that is in both files but I need to figure out which is not duplicate disk and not in use by other VG's on the systems.
I am doing this and not helping.
for X in $(cat disk.list)
do
grep ${X} new.disk.list
done
An "-v" option give me somethong different.
Appreciate your help.
Thanks....
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 03:13 PM
тАО05-13-2009 03:13 PM
Re: Help on removing duplicate in 2 files
You can use 'grep' and a file of the devices you want to sift:
Given that '/etc/lvmtab' is a binary file, we need to use 'strings'. You could do:
# strings /tmp/lvmtab|grep -v vg|grep -f /tmp/mydev
...and:
# strings /tmp/lvmtab|grep -v vg|grep -v -f /tmp/mydev
...where the /tmp/mydev file lists the devices one per line.
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 04:15 PM
тАО05-13-2009 04:15 PM
Re: Help on removing duplicate in 2 files
I got the file ready by filtering the output from powermt command but now I am trying to insert /dev/dsk in front of it using "sed".
Once, I have that done I think I will be in place.
Thanks for your help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-13-2009 06:30 PM
тАО05-13-2009 06:30 PM
Solution>for X in $(cat disk.list)
You need to use vector methods:
fgrep -v -f disk.list new.disk.list
This excludes all of the lines in disk.list from new.disk.list. (Other options are needed of some lines are substrings of other longer ones, add -x.)
>insert /dev/dsk in front of it using "sed".
sed 's:^:/dev/dsk:' file
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО05-15-2009 01:53 PM
тАО05-15-2009 01:53 PM