topic Re: Selecting Unique Values from many List in Operating System - HP-UX

Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 18:11:50 GMT

I have a question which is giving me a serious problem

I have like 19 different list which contains the name of the server but what I need is just unique ones.

First thing I need to do is just make a unique list within the list itself

i.e. delete anything that is repeated inside the list

like for example

in list1

i have

a

b

c

d

e

a

c

f

g

h

so i just need is a b c d e f g h i mean unique values which is easier anyone can do it i guess.

For the first list its ok

Like that i have 19 liist and i have to do that the same thing which is ok not that hard

the problem is now,

there should not be anything repeating in any one of those list

i.e. if something is there in the first list means that should not be in 2nd list and so on. so that means

for the first list its ok i dont need to do anything as it is after deleting the duplicates

second list I have to compare with the first list and delete if anything from the first list

third list i have to compare with first and second list and anything present in first or second list I have to delete from third list and so on upto 19 list

Plz can anyone suggest me how can i do that

let me know if anything confusing

just i want is not repeated in any one of those 19 list i hope thats pretty much clear

Thanks In advance

its serious problem im having

Re: Selecting Unique Values from many List

James R. Ferguson — Mon, 27 Apr 2009 18:19:45 GMT

Hi:

If you don't mind sorting and you want one list when done:

# sort -u file1 file2 file3 ...

or if the filenames are similar:

# sort -u f[0-9]*

Regards!

...JRF...

Re: Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 18:40:32 GMT

Thaks James but looks like you dint get what my problem is

I dont want one list i want all the 19 list but

there should not be anything repeating in any one of those list

i.e. if something is there in the first list means that should not be in 2nd list and so on. so that means

for the first list its ok i dont need to do anything as it is after deleting the duplicates

second list I have to compare with the first list and delete if anything from the first list

third list i have to compare with first and second list and anything present in first or second list I have to delete from third list and so on upto 19 list

Thanks

Re: Selecting Unique Values from many List

OldSchool — Mon, 27 Apr 2009 18:55:12 GMT

so, for each of the 19 files, you want to find only the unique entries *within* that file....

and then for each of those nineteen, you want to remove any entries which appear in the prior "x" files?????

such that when working with the 16th file, any entry in 16 which appears in 1-15 should be ommitted?????

"sort -u file1 > file1s" will get you the unique values and needs to be done for each file....

using those results, you can do:

grep -v -f file1s -f file2s -f file3s ....
-f file15s file16u > file16su

would remove any entries occuring the sorted/unique results of 1-15 from the 16, leaving only those unique to 16.

Re: Selecting Unique Values from many List

Steven Schweda — Mon, 27 Apr 2009 18:56:22 GMT

> [...] looks like you dint get what my
> problem is

He's not alone.

Perhaps you could show us what you want to
get, instead of trying to describe it.

Perhaps more blank lines would help.

Re: Selecting Unique Values from many List

James R. Ferguson — Mon, 27 Apr 2009 19:06:06 GMT

Hi (again):

OK, let's see if this gives you what you want:

# cat ./reduce
#!/usr/bin/perl
use strict;
use warnings;
my %seen;
while (<>) {
print;
chomp;
$seen{$_}++;
if (eof) {
print "------\n";
last;
}
}
while (<>) {
chomp;
print "$_\n" unless exists $seen{$_};
$seen{$_}++;
print "------\n" if eof;
}

...Run as:

# ./reduce f1 f2 f3 ...

For example, given:

# cat f1
a
b
c
d
e
f

# cat f2
a
c
f
g
h
i

# cat f3
a
f
h
i
k
l
m

# ./reduce f1 f2 f3
a
b
c
d
e
f
------
g
h
i
------
k
l
m
------

Regards!

...JRF...

Re: Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 19:16:00 GMT

Steven,

Oldschool got my problem, thats what I wanted to do.

Re: Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 19:41:16 GMT

James Thanks alot
it prefectly works but how can I find which output is for which file

I know its in the order I provided in the command line but

----is there any way I can have name of the file in the beginning----------

Thanks alot
really Great job

Re: Selecting Unique Values from many List

James R. Ferguson — Mon, 27 Apr 2009 19:47:13 GMT

HI (again):

Use this to trace the file names:

# #!/usr/bin/perl
use strict;
use warnings;
my %seen;
while (<>) {
print;
chomp;
$seen{$_}++;
if (eof) {
print "[ $ARGV ] ------\n";
last;
}
}
while (<>) {
chomp;
print "$_\n" unless exists $seen{$_};
$seen{$_}++;
print "[ $ARGV ] ------\n" if eof;
}

# ./reduce f1 f2 f3
a
b
c
d
e
f
[ f1 ] ------
g
h
i
[ f2 ] ------
k
l
m
[ f3 ] ------

...using the data I used in my previous post.

Regards!

...JRF...

Re: Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 19:51:36 GMT

It dint work for my list. I tried with just 2 and dint work so i dint try with many

cat uniq_from_many_list.pl
#!/usr/bin/perl
use strict;
use warnings;

my %seen;
while (<>) {
print;
chomp;
$seen{$_}++;
if (eof) {
print "------\n";
last;
}
}

while (<>) {
chomp;
print "$_\n" unless exists $seen{$_};
$seen{$_}++;
print "------\n" if eof;

}

cat USAGE3s
astro
bhpnbi2c
bhprtg13
bhprtg13
uhprtg6
uhprtg6
uhprtg6

cat MAFGGDBs
astro
benji
bhpmaf13
bhpmaf14
bhpmaf15
bhpmafg7
bhpmafg8
dhpmaf17
dhpmafg1
dhpmafg1
dhpmafg2
dhpmafg4
dhpmafg6
uhpmafg5

uniq_from_many_list.pl USAGE3s MAFGGDBs
astro
bhpnbi2c
bhprtg13
bhprtg13
uhprtg6
uhprtg6
uhprtg6
------
astro
benji
bhpmaf13
bhpmaf14
bhpmaf15
bhpmafg7
bhpmafg8
dhpmaf17
dhpmafg1
dhpmafg1
dhpmafg2
dhpmafg4
dhpmafg6
uhpmafg5
------

Anything need to be changed so that it will work ?

Re: Selecting Unique Values from many List

James R. Ferguson — Mon, 27 Apr 2009 20:05:54 GMT

Hi (again):

If you have leading and or trailing whitespace in any line, the original code would consider this significant. This would account for the failure with your data. we can remdedy this:

# cat ./reduce
#!/usr/bin/perl
use strict;
use warnings;
my %seen;
while (<>) {
print;
chomp;
s/^\s*//; s/\s*$//;
$seen{$_}++;
if (eof) {
print "[ $ARGV ] ------\n";
last;
}
}
while (<>) {
chomp;
s/^\s*//; s/\s*$//;
print "$_\n" unless exists $seen{$_};
$seen{$_}++;
print "[ $ARGV ] ------\n" if eof;
}
1;

Regards!

...JRF...

Re: Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 20:51:07 GMT

Great!!!!!!!
Thank You So Much

ITs working like i wanted

Re: Selecting Unique Values from many List

pareshan — Mon, 27 Apr 2009 20:51:57 GMT

I have found the solution