Archivery Tricks

[Ralph Grothe]

Hi,

this is a silly little problem I have.
I want to transfer a directory tree between two hosts.
Now with the this tar command I have always been a happy camper:

# cd /opt/gnu
# tar cf - mysql | ssh other_host \(cd /opt \; tar xvf - \)

But this time I needed to omit a submount on /opt/gnu/mysql/var.
So I thought about using find together with cpio, and I tried this

# cd /opt/gnu
# find -depth -xdev mysql | cpio -o | ssh other_host \| cpio -imuxdav /opt

Somehow it say
51093 blocks
51093 blocks
but actually wrote nothing on other_host.

Where is it gone?
What's wrong with my cpio syntax?

[Ralph Grothe]

ah I saw the lonely pipe was wrong on ssh other_host, so I issued this instead:

# find mysql -depth -xdev|cpio -o|ssh other_host cat - \|cpio -imuxdav /opt

but didn't solve it...

[Stefan Farrelly]

Try inserting a dd around the ssh; dd bs=64k | ssh .. | dd bs=64k | cpio ....

[Frank Slootweg]

It is probably best to use rcp(1), with the -r and -p options, instead of things like tar and cpio.

Like Stefan implies, 'remote pipes' (i.e. pipes used with remsh(1), ssh, etc.) often have buffering problems, which can be fixed by dd(1) around the remote pipe, but it is still tricky to get things right.

[Wodisch]

Hello Ralph,

if you are using Secure SHell (ssh), why not use Secure
CoPy (scp)? Or even "rsync" with "ssh" as transport?
Then you can avoid all that "tar" and "cpio"...

Just my ?0.02,
Wodisch

[Ralph Grothe]

Hello all,

the reason why I don't want to use rcp, scp, or anything which does recursive copying is simply that those will follow links instead of duplicating them as what they are.
The same I think goes for any other special files (e.g. devices, fifos, domain sockets etc.) which cannot be treated correctly by above commands.

I will come back later to assign points...

[harry d brown jr]

Grab the gnu-tar, it allows you to tar from one machine to another (even remote devices), and it has an option to give it a list of files/directories to back up (the -T ):

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.22/

[Frank Slootweg]

If you indeed need to copy 'special' things, then tar is also mostly out. I think in that case only cpio (and it's pax equivalent), fbackup, etc. are useable.

For the cpio case, use the B option to get a 5KB block size and let dd(1) read from cpio and write to cpio with a 5k block size, i.e. "cpio -oB .... | dd ibs=5k obs=..." and
"dd ibs=... obs=5k | cpio -iB ...". The reason for this is that 'remote pipes' do not quarantee which 'chunks' they will use, i.e. when you write 10 times 5k to a remote pipe, you will get 50k out of it, but not neccessarily in 10 blocks of 5k. The dd constructs solve that problem.