Speed differences archiving filesystems, ssh/remsh vs. nfs

[Ralph Grothe]

Hi all,

I have to move some 20 GB of data between hosts, and I was wondering if there are any speed differences between transfers either via nfs, or pipe via remote shell commands (e.g. ssh/remsh combined with tar, cpio, dd etc.)
The connection is a 100 MBit/sec Ethernet.
(let alone in-pipe compression, just a rough estimate needed,
ok it depends on current network load, so just a qualitative answer is expected)

Regards
Ralph

[harry d brown jr · Accepted solution]

Ralph,

ftp and rcp (scp) will be MUCH faster than NFS.

live free or die
harry

[Ralph Grothe]

Hi Harry,

that's what I expected, since NFS uses this meta data type representation (forgot the exact term), and thus seems to me to have much more overhead.

[Steven Sim Kok Leong]

Hi,

To add on, from my experience, scp is slower than either rcp or ftp because of the encryption overhead, despite the compression.

Hope this helps. Regards.

Steven Sim Kok Leong

[Darrell Allen]

Hi Ralph,

As Steven says, ftp and rcp will be faster than scp or sftp because of the encryption overhead. With 20GB the difference could be very significant, especially on a box with cpu restraints (slower processors, single processor, heavily loaded system).

Darrell

[Volker Borowski]

It depends,

some switches / routers are configured to throttle down bandwidth for ftp-traffic.

I found "tar -cf- . | remsh .... tar -xf - "
to be the speediest one, esp. as you can create several streams in parallel.

(... but I got kicked when the network guys did find out :-)

Hope this helps
Volker

[Ralph Grothe]

Hi Volker,

that way I have done data transfers between hosts so far as well.
My only modification (despite the extra overhead for encryption, btw. when the session key once has been created ssh uses a much quicker symmetric encryption algorithm unlike the more secure assymetric during key negotiation) is that I still prefer ssh.

This way I usually do it:


# cd /source_tree
# tar cf - . | ssh remote_host \(cd /target_dir\; tar xf -\)

I think as root you can neglect tar's -p flag.
Usually I start this as a batch command sequence (nohup would also do), so that the process isn't associated with a terminal (cave SIGHUP).