topic Re: remove duplicate entries in Operating System - HP-UX https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572873#M831233 hi,<BR /><BR />i tried this:<BR />==============<BR />$ more 1.txt<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />$ uniq 1.txt 2.txt<BR />$ more 2.txt<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />$<BR /><BR /><BR />regards<BR />yogeeraj Wed, 29 Jun 2005 04:15:40 GMT Yogeeraj_1 2005-06-29T04:15:40Z remove duplicate entries https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572870#M831230 Hi,<BR /><BR />I am doing some batch processing of a file that I dump from NNM. I only one to keep one unique entry based on the IP address. For example I need to run the following<BR /><BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR /><BR />into<BR /><BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical Wed, 29 Jun 2005 03:55:06 GMT https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572870#M831230 kholikt 2005-06-29T03:55:06Z Re: remove duplicate entries https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572871#M831231 uniq old_file new_file<BR /><BR />Mark Syder (like the drink but spelt different) Wed, 29 Jun 2005 04:03:24 GMT https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572871#M831231 MarkSyder 2005-06-29T04:03:24Z Re: remove duplicate entries https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572872#M831232 You can use like,<BR /><BR /> sort -u <FILENAME><BR /> <BR />hth.<BR /></FILENAME> Wed, 29 Jun 2005 04:12:12 GMT https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572872#M831232 Muthukumar_5 2005-06-29T04:12:12Z Re: remove duplicate entries https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572873#M831233 hi,<BR /><BR />i tried this:<BR />==============<BR />$ more 1.txt<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />$ uniq 1.txt 2.txt<BR />$ more 2.txt<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />$<BR /><BR /><BR />regards<BR />yogeeraj Wed, 29 Jun 2005 04:15:40 GMT https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572873#M831233 Yogeeraj_1 2005-06-29T04:15:40Z Re: remove duplicate entries https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572874#M831234 Hi<BR /><BR />Use the "-d" option with "uniq" to list one copy only of each repeated line in the input file.<BR /><BR />$ more test<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR /><BR />$ uniq test<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR /><BR />Note: There are 2 entries for "www05_bck.abc.com "<BR /><BR />$ uniq -d test<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR /><BR /><BR /><BR />Rgds<BR />Shyjith Wed, 29 Jun 2005 05:43:21 GMT https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572874#M831234 Shyjith P K 2005-06-29T05:43:21Z Re: remove duplicate entries https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572875#M831235 You can also try as,<BR /><BR />-- testfile--<BR />your inputs<BR /><BR /># sort -mu testfile<BR />www05_bck.abc.com ; 1.23.32.35 ; Critical<BR />uat01.abc.com ; 1.23.4.36 ; Critical<BR />uat01_ap1.abc.com ; 1.23.5.36 ; Critical<BR />uat01_ap2.abc.com ; 1.23.6.36 ; Critical<BR />uat01_bck.abc.com ; 1.23.32.36 ; Critical<BR />hissta01.abc.com ; 1.23.4.37 ; Critical<BR />hissta01_ap1.abc.com ; 1.23.5.37 ; Critical<BR /><BR />hth. Wed, 29 Jun 2005 08:31:00 GMT https://community.hpe.com/t5/operating-system-hp-ux/remove-duplicate-entries/m-p/3572875#M831235 Muthukumar_5 2005-06-29T08:31:00Z