Verify two files are exactly equal eachother.

[Edgar Brito]

I made a copy of a file. How can I compare them and asure that they both are equal one to the other?

[Pete Randall]

Run diff:

diff file1 file2

man diff for details.


Pete

[Ivan Krastev]

use cksum - http://docs.hp.com/en/B2355-60105/cksum.1.html


regards,
ivan

[Dave Hutton]

sum or cksum (sum I think is being obsoleted)

sum filea fileb
cksum filea fileb

[Robert-Jan Goossens]

Hi,\

cksum -- cksum is typically used to verify data integrity when copying files between systems.

Regards,
Robert-Jan

[James R. Ferguson]

Hi:

As noted, 'diff' for ASCII (text) files. For binary files you can use 'cksum' or for a bigger, better hammer, use 'md5'.

Regards!

...JRF...

[Hein van den Heuvel]

I the files are available from a single system, then 'diff' is the only and right way to go. Binary files as well as text files.
Why 'risk' a false positive from a checksum?

For files on different systems, perhaps even with a different platform, a checksum tool like cksum or md5 is appropriate.

Food for thought: When is the last time you, or anyone out here ever saw a file being different after a succesful copy?
I appreciate the 'check and double-check' attitude, but in reality a simple byte-count (ls -l) will probably catch 99.9999% of the cases. My advice? Don't bother!

And if you wanted to check seriously then you must make sure that the (OS) and controller caches are flushed because a compare against recently copied files might not read anything from the disk itself, relying on cached data. So if it misteriously was read wrong the first time, then the second time the same wrong data could be used.

Enjoy!
Hein.

[TwoProc]

Hein,

Re: last time I saw...

On an Intel box? Often enough to worry about it.

On an HP box? Can't even remember a case...

[James R. Ferguson]

Hi (again):

> Hein: If the files are available from a single system, then 'diff'...Binary files as well as text files.

Ah, thanks, I never remember that 'diff' doesn't care.

> Hein: Why 'risk' a false positive from a checksum?

While I would consider that very unlikely, that's why I suggested 'md5sum'.

> Hein: For files on different systems, perhaps even with a different platform, a checksum tool like cksum or md5 is appropriate.

This assumes that the systems share a common newline indication for text files. If the end-of-line characters are different (e.g. UNIX vs. Windows or MAC: \012 versus \015\012 versus \015, respectively) then this is going to alter the results.

> Hein: And if you wanted to check seriously then you must make sure that the (OS) and controller caches are flushed...

Hmmm. that is interesting.

Regards!

...JRF...

[Steven Schweda]

> diff file1 file2

> use cksum

If you care only _whether_ they're different,
but not about what the differences are, then
"cmp" can be much nicer than plain "diff".

"cmp" appears to quit when it finds the first
difference, while "diff" can waste a lot of
time processing (large) different files.
Similarly, a checksum calculation always
requires reading all of each file.

Also, a lame "diff" program can require you
to use "> /dev/null" to avoid confusing your
terminal when dealing with non-text files.

> When is the last time you, or anyone out
> here ever saw a file being different after
> a succesful copy?

It's not very hard to get a partial FTP or
HTTP download which fails with no symptom
other than a corrupt (partial) file.
Sometimes it's the server's problem,
sometimes the client's.

[Hein van den Heuvel]

[Dang, have to re-enter. Got the bogus "Due to the presence of characters known to be used in Cross Site Scripting attacks, access is forbidden. This web site does not allow Urls which might include embedded HTML tags.". Balony! (sp?). Normally I have a copy in a paste buffer, but not this time.]

Steven wrote> "cmp" appears to quit when it finds the first difference, while "diff" can waste

Ah! Thanks for that.
This is why I keep reading even seemingly trivial questions.
Someone might point out a new solution, or at least remind me of long since forgotten alternatives.

Hein> ever saw a file being different after a succesful copy?
Steven> It's not very hard to get a partial FTP or

That's why I wrote copy, not transfer.
And I suppose copies with NFS involved are a little suspect also.
it's all too easy to get

Hein> (OS) and controller caches are flushed...
JRF> Hmmm. that is interesting.

Well, for a good sized file (1GB or more) the tails of the files with chase the heads out of the caches. But file of even a few 100 MBs might just fit in the cache, unless the cache is trained to recoginize long sequential IO patterns (iirc the Tru64 UBC special cased this).

So you really want to at least issue 'sync' before a compare. The sync might actually trigger a report for IO errors for a long since (30 seconds :-) completed copy.

Cheers,
Hein.

[Dennis Handly]

>JRF: For binary files you can use 'cksum'

As Hein says, you can also use diff(1) but you'll need to suppress the nasty warning when different.

For Steven's cmp(1), you'll want to use -s to suppress the output.

>Hein: [Dang, have to re-enter.

With firefox, just go back and resubmit.