topic Re: Archivery Tricks in Operating System - HP-UX

Archivery Tricks

Ralph Grothe — Fri, 05 Oct 2001 09:39:20 GMT

Hi,

this is a silly little problem I have.
I want to transfer a directory tree between two hosts.
Now with the this tar command I have always been a happy camper:

# cd /opt/gnu
# tar cf - mysql | ssh other_host \(cd /opt \; tar xvf - \)

But this time I needed to omit a submount on /opt/gnu/mysql/var.
So I thought about using find together with cpio, and I tried this

# cd /opt/gnu
# find -depth -xdev mysql | cpio -o | ssh other_host \| cpio -imuxdav /opt

Somehow it say
51093 blocks
51093 blocks
but actually wrote nothing on other_host.

Where is it gone?
What's wrong with my cpio syntax?

Re: Archivery Tricks

Ralph Grothe — Fri, 05 Oct 2001 09:45:39 GMT

ah I saw the lonely pipe was wrong on ssh other_host, so I issued this instead:

# find mysql -depth -xdev|cpio -o|ssh other_host cat - \|cpio -imuxdav /opt

but didn't solve it...

Re: Archivery Tricks

Stefan Farrelly — Fri, 05 Oct 2001 09:59:31 GMT

Try inserting a dd around the ssh; dd bs=64k | ssh .. | dd bs=64k | cpio ....

Re: Archivery Tricks

Frank Slootweg — Fri, 05 Oct 2001 10:30:43 GMT

It is probably best to use rcp(1), with the -r and -p options, instead of things like tar and cpio.

Like Stefan implies, 'remote pipes' (i.e. pipes used with remsh(1), ssh, etc.) often have buffering problems, which can be fixed by dd(1) around the remote pipe, but it is still tricky to get things right.

Re: Archivery Tricks

Wodisch — Fri, 05 Oct 2001 11:08:14 GMT

Hello Ralph,

if you are using Secure SHell (ssh), why not use Secure
CoPy (scp)? Or even "rsync" with "ssh" as transport?
Then you can avoid all that "tar" and "cpio"...

Just my ?0.02,
Wodisch

Re: Archivery Tricks

Ralph Grothe — Fri, 05 Oct 2001 11:13:29 GMT

Hello all,

the reason why I don't want to use rcp, scp, or anything which does recursive copying is simply that those will follow links instead of duplicating them as what they are.
The same I think goes for any other special files (e.g. devices, fifos, domain sockets etc.) which cannot be treated correctly by above commands.

I will come back later to assign points...

Re: Archivery Tricks

harry d brown jr — Fri, 05 Oct 2001 12:22:56 GMT

Grab the gnu-tar, it allows you to tar from one machine to another (even remote devices), and it has an option to give it a list of files/directories to back up (the -T ):

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.13.22/

Re: Archivery Tricks

Frank Slootweg — Fri, 05 Oct 2001 12:34:25 GMT

If you indeed need to copy 'special' things, then tar is also mostly out. I think in that case only cpio (and it's pax equivalent), fbackup, etc. are useable.

For the cpio case, use the B option to get a 5KB block size and let dd(1) read from cpio and write to cpio with a 5k block size, i.e. "cpio -oB .... | dd ibs=5k obs=..." and
"dd ibs=... obs=5k | cpio -iB ...". The reason for this is that 'remote pipes' do not quarantee which 'chunks' they will use, i.e. when you write 10 times 5k to a remote pipe, you will get 50k out of it, but not neccessarily in 10 blocks of 5k. The dd constructs solve that problem.