Copy sizes are different

[N.D]

I am copying from one filesystem to another;
' cp -rp /source_dir /destination_dir '

but when i do a bdf the sizes are different and so is the file sizes i have tried recopying, deleting etc, but still the same and i need to make sure the data copied over is identical

[David_246]

Hi,

Better is to use tar for these occasions. Copi-ing directories may resume in less data. For each file a specific amount of space is used per file. do an ls -lad of all directories in both directories. This shows you the difference.

Also cp does not copy special files. tar does (not all, but read the man page for those).

So :

tar cvf - . | ( cd /to/new/dir; tar xvf - )

Of course you inittiate this command from the source directory.

Regs David

[Robert-Jan Goossens]

Hi,

You can also use cpio,

# find /source_dir | cpio -pcmudv /dest_dir

Hope it helps,

Robert-Jan.

[Stefan Farrelly]

If you want to be sure the copy sizes are identical then tar up the source files first, creating a single tar file which you can then run sum on - to get a checksum value, then ftp to remote destination, run sum and confirm the same which means they are identical, then untar it.

[Duncan Edmonstone]

Hi,

Are you sure its not just the directorys that are different sizes?

Directory files grow, but never shrink, so when you get more than a certain number of files in a directory, the directory itself takes up slightly more space, but if some of the files in the dir are deleted it doesn't get smaller again.

A better way of checking whether your copy was successful is to look at just the files. Try this:


cd /source_dir
find . -type f -exec ll {} \; > /tmp/source

cd /destination_dir
find . -type f -exec ll {} \; > /tmp/destination

diff /tmp/source /tmp/destination

If this reports no differences your OK.

HTH

Duncan

[Elena Leontieva]

To verify the data integrity, compare checksum and size:

cksum filename

Elena.

[A. Clay Stephenson]

You are almost certainly a victim of a "sparse" file.

Here is how 1MB file that occupies on two bytes can be created.

char buf = 'x';
fdes = open("myfile",O_RDWR | O_CREAT,0664);
write(fdes,&x,1);
lseek(fdes,1000000,0);
write(fdes,&x,1);
close(fdes);

This creates "myfile" writes "x" at the beginning of the file, skip 1000000 bytes, and writes a second "x" --- a sparse file.

This file would ls as 1000000 bytes but bdf as two blocks --- the intervening bytes are never allocated BUT when myfile is copied those intervening bytes will be padded with NUL's on the copied file and instantly bdf will report very different numbers. When directories are copied, the "holes" (artifacts of deleted files in that directory) are not copied so that directories commonly don't copy exactly.

bdf and ls are not very good tools to determine if a file has copied correctly. After all, the SIZE of the files may be identical but that says nothing about the CONTENTS of the file. A much better test is to run cksum of each file.

Here is one method:

On the source dir:
find . -type f -exec cksum {} \; | sort > srclist
Now on the destination:
find . - type f -exec cksum {} \; | sort > destlist

diff srclist destlist -- if any lines are reported you have problems.

[Sundar_7]

Couple of things missing.

char buf = 'x';
FILE *fdes;
fdes = open("myfile",O_RDWR | O_CREAT,0664);
write(fdes,&des,1);
lseek(fdes,1000000,0);
write(fdes,&des,1);
close(fdes);