Need a unique Diff command

[Belinda Dermody]

Have a situation where we receive a full database dump each night 500+m from a client and for us to process it takes a long time and we have to break replication,
So my question is there a diff command that will ONLY give us the lines that are diff in the NEW file only from the old file or not in the old file. So we only have to work with any changes or additions.

[Pete Randall · Accepted solution]

James,

Wouldn't "diff file1 file2 |grep '>' " give you what you want?


Pete

[harry d brown jr]

What would that database be?

What kind of storage is the database on?

What kind of backups are you doing?

live free or die
harry

[Hein van den Heuvel]

Look like you could try: comm -13


Hein.

[Belinda Dermody]

Thanks for the information, the diff with the grep completely slip my mind, I am running it now as a test to see how long it takes and if if gives the database people the needed info.

It is a sybase database dump to a flat file and zipped up for us. We maintain a database for them and they haven't gotten into the update change only part yet, so all they can do is provide a full dump to a flat file and we turn around process it. I am just the unix guy who is in charge of the sftp site and provide the nightly file to our database people to process and I am just trying to figure out a shorter process to do this.

The comm -13 gives me the two sets of line the new and the old, I just one lines from the new file

[H.Merijn Brand (procura]

comm only works on sorted files

diff file_old file_new | perl -ne's/^> // and print'
or
diff file_old file_new | grep '^>' | sed 's/^> //'

would give you the new lines only

Enjoy, Have FUN! H.Merijn