cp -pr doesn't copy entire dir

[Richard Quinn]

Hi,

I am trying to copy directories from one lv to another using the cp command ( -pr params). The directories appear to have moved ok and it's only when I check using du -k that I notice that the size of the old and new dirs are different. Can anyone explain why this is happening or what I can do to resolve it.

Thanks for your help,
Donal

[Ian Dennison_1]

whats the full command you are using?

'cp -rp *' will not get .profile, .login, etc.

What User are you using? root?

Share and enjoy, Ian

[Deepak Extross]

Maybe your user-id does not have permissions on some of the files in the source directory? Make sure you're logged in as root.

[Sebastien Masson]

Use this:

cd /source
find . -depth | cpio -pvmdu /destination/.

[harry d brown jr]

Why would you expect a "du" to be the same? Remember, you copied existing directories and files to a new location, so the new copy requires less space. Example is directories: as files come and go the directory "size" can grow and grow, until you recreate it. What is more important, is that you can be assured that "cp" worked!

live free or die
harry

[Robin Wakefield]

Hi Donal,

You could assure yourself that everything's copied across by running:

find directory ! -type d | xargs ll | awk '{TOT=TOT+$5}END{print TOT}'
find directory | wc -l

on the old and new directories and compare.

Rgds, Robin

[Wim Rombauts]

I agree with harry.

The source directories contain "empty spaces" where removed files once were registered. The target directories don't contain these empty spaces.

If cp doesn't return an error, it has worked fine.
If you insist to verify this, do the following :
find | wc -l
find | wc -l
If both instructions return are the same number, every file and directory has been copied.
Detailed verification can be done with the following instructions :
find -exec ll -d {} \; | cut -c 35-44,58- > file1
find -exec ll -d {} \; | cut -c 35-44,58- > file2
diff file1 file2
This returns you all differences is files and filesizes (including directorysizes).

[Deepak Extross]

Hi,
Why dont you try to locate missing files, if any.
Try this:
#tar cvf /tmp/d1.tar
#tar tvf /tmp/di.tar | wc -l
(this will give the count of all files and directories in , including hidden files)
The count displayed on your source and destination should be same - if not, investigate the missing files.
If they are indeed the same, then you can have a warm fuzzy feeling that cp has done its job.

[Roger Baptiste]

hi,

Directories size can vary, because the files in the new directories would not have the fragmentation which would be there in the files of the old directory. i.e the new directories should generally show LEsser space used than the old ones.

To copy directories , i normally prefer
the cd $SOURCE ;find . | cpio -pdlmuv $DEST ;
(easy to remember -> pdlmuv is PaDdLeMUV)

HTH
raj

[Richard Quinn]

Thanks for all the help. I am now a trusting friend of cp again.

Thanks again,
Donal

[Darrell Allen]

Hi Donal,

Here's my 2 cents on the very good comments already made:

I formerly used:
tar cvf - . | (cd /new_dir; tar xvf -)

Then I read this reply by Bill Hassell:
The fastest way is to use the puddlemove options in cpio as in:
$ cd /source_dir
$ find . | cpio -pudlmv /destination_dir

Bill is right. cpio is much faster than tar.

About the du differences: as the number of files increase in a directory, the size of the directory entry must increase as well. It's kind of like filling up a sheet of paper with a list and having to add another sheet. That's why a newly created empty directory is smaller than one that has many files in it. The directory size does not decrease even after removing all files. You can erase lines from your list on the sheets of paper but you still have the same number of sheets. When you copy that directory, the destination directory is created small again and grows according to the number of files you have. Just like copying the list by hand, you start with a fresh sheet of paper and don't copy erased lines.

Make sense?

Darrell

[Richard Quinn]

Thanks Darrell, clear explanation. By the way does cp copy hidden files also?

[Robin Wakefield · Accepted solution]

Hi Richard,

It copies hidden files unless you tell it not to, e.g.

cp /dir/* /dir1 will not
cp -r /dir /dir1 will

Rgds, Robin

[Bill Hassell]

JUst to complete the treatise on whu a copied directory sgtructure is not the same size: the copy will be the same size, or larger or smaller. Never use the size of the directory structures as confirmation of a good copy. Always count the number of files and the number of directories as in:

find -type d | wc -l
find -type d | wc -l
find -type f | wc -l
find -type f | wc -l

The file count should be the same, the directory count might larger by 1 if the destination is a new volume due to lost+found directory.

The reason for smaller szes has been dicussed already (directory entries remain after the files are removed), but what if the destination is much larger?

The answer is: sparse files. These are files that are created with 'holes' in them. Consider creating a new file by writing record #1, then by using lseek, write record #1 million, then close the file. The rest of the intermediate records are never written and the directory will track just those two records.

However, Unix (it's friendly this way) will allow you to read this file equentially, but all the missing records will be full of zeroes. And this is what happens when the file is copied. So a 2 record file now becomes a 1 million record file and the destination is much larger. An common example of a sparse file is the core file.