Problems copying hidden files

[Admin32]

Greetings all and Happy new year !

I've been trying to copy a whole dir from one drive to another using cp, but I cannot manage to copy the hidden files.

The dir is /home/projects/ and I want to copy it to a temp dir /mnt/sdc2.
Both partitions are ext2 formatted.

I;ve tried cp -d -a -b but had not luck copying the hidden ".xxxxx" files.

Any idea how I can overcome this problem ?

Using Tar is not an option since these dir's are well over 3 gigs.

Cheers,

[Vincent Fleming]

find /home/projects -depth -print | cpio -pdmvu /mnt/sdc2

always works for me.

Good luck,

Vince

[Andrew Cowan]

Hidden files?

Are you sure that you are root/the correct user?

[Stuart Browne]

Hidden.

Go to a directory, issue "ls -l". Now issue "ls -al". See the difference?

By default, the 'ls' command (and I suppose most other shell utilities) treat files that start with a period (.) as 'hidden'. Things like 'samba' purposely ignore them (unless told not too) so that they cannot be used.

More often than not, they are for storing configurations, or temporary data (for instance, Netscape (mozilla) uses .mozilla/ for all of it's settings, and local caches).

Issuing 'cp * /blah' will only copy files which do not begin with a period. To copy all of those as well, you need to issue something similar to 'cp * .??* /blah'. The ".??*" will grab all files/directories starting with a period, but not the special "." and ".." directories.

Hope this helps explain the situation a bit better for Andrew.

[Vincent Fleming · Accepted solution]

Stuart,

Just as an FYI, using find and cpio is superior to using "cp" for several reasons... first, "find" will get every file, where "cp" depends on the wildcards you issue, and "cpio" will restore the user and group id, file permissions, and modification times so the the file is an identical copy of the original, down to the file permissions and ownership, where "cp" will not.

Cheers,

Vince

[Stuart Browne]

Yah.. I kno.. Use 'em myself.. Is much easier when copying multi-GB file-sets with thousands of files :P No need to re-set permissions/ownerships.

But sometimes, simple is best (populating a home directory for instance. Most permissions and ownerships need to be reset anyway).

[Admin32]

Vince,

Thank you for the answer.
While it worked, there is a small problem.

when issuing find /home/projects -depth -print | cpio -pdmvu /mnt/sdc2 I get the following dir structure created : /mnt/sdc2/home/projects/

I need to keep the structure to this: /mnt/sdc2/'contents of projects dir'

So that when someone enters the sdc2 dir, they see exactly what the /home/projects/ dir contains.

I tried find * -depth -print | cpio -pdmvu /mnt/sdc2 from inside the /home/projects dir, but it failed to copy the hidden files.

Any suggestions ?

p.s I tried to understand what all the -pdmvu flags mean, but i found it very confussing, if you can quickly explain what each flag does, i would appreciate it.

Thanking you, and everyone who responded.

Cheers,

[Kelli Ward]

Hi,
This should help.

cd /home/projects
find . -depth -print | cpio -pdmvu /mnt/sdc2

It will bypass the absolute path and copy relatively.

-pdmvu
p - pass (pass through output will send the files to /mnt/sdc2)
d - directories (create as needed)
m - modification (retain all modification times)
v - verbose (tell me everything your [the command]doing)
u - unconditionally (copy files unconditionally)

HTH,
Kel

[Kelli Ward]

Also,
Are you sure the hidden files weren't copied?

I am going to make an assumption, I apologize if I'm wrong. :)

If you're used to using HP-UX, ls and ll will list hidden files automatically, if you're the root user. If you are a "normal" [whatever that means] ;) user, you need ls -a or ll -a to list all files.

Many Linux flavors, when used generically, will not show hidden files, even to root. Some linux versions don't even have the ll command at all and you are forced to use (if not aliased) ls -l (or to see hidden file ls -la)

Try using ls -la and see if your missing files show up.

To try to emulate your problem, I did test the cpio command I wrote in my previous reply on my Debian system and it worked perfectly for me, including copying the hidden files.

Let me know if ls -la (or ll -a, if you have it) scares them up for you.

Kel

[Kelli Ward]

And..
Man, I wish I'd learn to read, long day.

When you went into the /home/projects directory and used find * , you specified all files, but hidden files are a little different and will not be seen by find when just the * wildcard is used.
The next logical conclusion might then be to use find .* instead.

Be careful, because two "files" you easily can't see are . (current directory) and .. (parent directory), .* can also look at .. and start doing things in the parent directory.

I was inadvertantly nailed by this once when I was in a user directory and was changing file ownerships. I used chown user:group .* and the command did just what I told it to change all the hidden files, then move up one directory and do it there too. Oops!

A co-worker thought DOS and used rm *.* which did similar things but with disasterous results. (It wasn't me, honest.)

The best bet is to cd into the desired directory and use find . (Which tells find to do it's work from the current directory)

You can give me 0 for this reply, it should have been part of the last.

Thanks,
Kel

[Jochen Heuer]

Hi,

if you want to copy even the hidden files / directories just do

$ cd /dir; cp -r `ls -A` /new/dir

If you want to preserve as much permissions as possible use -a also:

$ cd /dir; cp -a `ls -A` /new/dir

Explanation:

'ls -A' lists all files (even the hidden ones) but not the directories . and ..

Regards,

Jochen