topic Re: Script help needed in Operating System - Linux

Script help needed

TheJuiceman — Tue, 17 Jul 2007 17:53:41 GMT

Hey gang,
This is an easy one, but for some reason it escapes me. This is what I need to do...

I have FileA that has a long list of miscellaneous entries

I have FileB that has a partial list of what FileA has in it.

I need to create FileC that has what is in FileA less FileB. How is the best way to pull this off? Thanks again.

Re: Script help needed

James R. Ferguson — Tue, 17 Jul 2007 18:10:06 GMT

Hi:

If the files are sorted (or can be) then you can use 'comm' [see its manpages]:

# comm -23 fileA fileB > fileC

Regards!

...JRF...

Re: Script help needed

KapilRaj — Tue, 17 Jul 2007 18:13:00 GMT

cat FileA | grep -vf FileB > FileC

Regds,

Kaps

Re: Script help needed

James R. Ferguson — Tue, 17 Jul 2007 19:01:38 GMT

Hi (again):

...and if for whatever reason your files are not sorted or cannot be sorted, use:

# cat ./filter
#!/usr/bin/perl
use strict;
use warnings;
my $fileA = shift or die "Usage: FileA FileA\n";
my $fileB = shift or die "Usage: FileA FileB\n";
my %line;
my @list;

open( FH, "<", $fileB ) or die "Can't open $fileB: $!\n";
while () {
$line{$_}++;
}
close FH;
open( FH, "<", $fileA ) or die "Can't open $fileA: $!\n";
while () {
push @list, $_ unless exists $line{$_};
}
print for @list;
1;

...run as:

# ./filter fileA fileB

Regards!

...JRF...

Re: Script help needed

TheJuiceman — Tue, 17 Jul 2007 20:32:21 GMT

Awesome. Thanks everyone!!!

Re: Script help needed

Dennis Handly — Tue, 17 Jul 2007 22:14:58 GMT

>Kaps01: cat FileA | grep -vf FileB > FileC

No need to use cat:
$ grep -vf FileB FileA > FileC

You'll have bad performance if the number of lines in FileB and FileA are very large.

A caution about grep, if FileB has lines are substrings of other lines in FileA, you'll miss them. You can fix this with grep -x.