NFS vs. SFTP which one faster?

[Gary L]

Hi

My boss asked me a question about data transfer from one HP-UX server to another HP-UX server, which way (speed/rate)is faster? Using NFS remote mount point "local" copy or SFTP from one to another? How about FTP?

Thanks a lot. Any answers will be very appreciate.

-G

[Mark McDonald_2 · Accepted solution]

Hi G

Would they be over the same network interface? Here I can sneak a few sftp's over the gigabit backup network in the day time, but they would never allow an NFS mount over it.

What size files?
How many files?
How often?

FTP is generally frowned up on as being less secure. However if you are inside of your firewalls, and the data is not too sensitive then its an option.

Mark

[Gary L]

Hi Mark

Thanks a lot for your faster reply, through NFS or SFTP? Just a kidding.

The total size around 100-150GB. It's a daily job. How many files? not sure, just know it's some oracle data files.

NIC are all std 100MB mode, they are all located inside firewall internal environment.

Which one is faster?

-G

[Mark McDonald_2]

100 - 150 Gb is quite large!!!

sftp will have lower overhead and therefor be quicker. Also the fact that it is a daily job, means there is no need for a permanent NFS connection.

One other idea for large files is tar to std out and remsh to the remote host:

tar cvf - filename | remsh hostname "cd /dir ; tar xvf -"

Good luck, other users might be able to make other suggestions.

Mark

[Gary L]

Thanks Mark.

Have a good day.

-Gary

[Dave Olker]

If the bulk of your 100-150GB of data is not changing every day you might also want to look into rsync. That allows you to push only the changes between the source and target. If not much changed that day, you may be able to save yourself a lot of network traffic, which can be very important over a 100MB link.

Dave

[rick jones]

Unless you are running NFS over IPSEC, I would think the proper comparison would be NFS vs plain FTP as SFTP/scp will have encryption overhead not present in NFS/TCP/IP.

The answer is, of course "it depends" :)

Ignoring disc performance on the premis it affects both equally (assuming NFS read rather than write)...

NFS performance will depend on the size of the effective TCP windows and the number of outstanding NFS requests per mount and the number of TCP connections per mount. How those are changed will have to come from the likes of Dave Olker or another NFS guru.

FTP of a single file will depend on the size of the effective TCP window. FTP will otherwise shove as much into the socket at one time as it can. Socket buffer settings are controlled in ftp on ux via (iirc) a -B option - and IIRC FTP will always make explicit setsockopt() calls and not rely on the the system defaults so you need to check the ftp/ftpd manpage(s).

Attached is some TCP performance boilerplate which is a work-in-progress. It might have a slight Linux bias presently but the broad brush strokes should transfer pretty well.

[Gary L]

Hi Dave and Rick

Thanks a lot for your good suggestions.

Have a good day.

-Gary

[OldSchool]

100BT will only get you, at absolute best, 42GB/hour, and probably less, so you're looking at a 2-5 hour transfer window. I doubt you want to have ftp up and logged in that long.

NFS would probably be a better choice I would think.

[rick jones]

NFS won't be any more secure than the FTP really. It will be a 2-5 hour TCP connection as well. Perhaps the nature of NFS traffic makes it more likely to notice a connection hijack, but if that is really a concern then Gary should be using IPSEC or the like.

[Gary L]

Thanks OldSchool.

Thanks Rick again.

Yes, currently, this data transfer job spends around 3 or more hours per day. It was set in crontab job and started at 4:00 a.m., it looks like a little bit long, so that's the resson why I asked NFS vs. SFTP which one is faster.

Queston for Rick, could you provide me more details about IPSEC, like how to set it and how to get this ... is it network side or HP-UX server side? "running NFS over IPSEC..."

thanks.

-Gary

[rick jones]

I am afraid I do not know the specifics of enabling IPSEC - one or more of the UX or networking manuals at https://support.hpe.com/hpesc/public/home  should discuss it. I take it you are making these transfers over a network not entirely under your control?

[Gary L]

searching IPSEC ...

Thanks a lot.

Have a good day.

-G

[Dave Olker]

If security is a concern and you're running 11i v3 you can use Secure NFS. That leverages Kerberos security and allows you to either enforce:

o Kerberos Authentication
o Send & Receive Data Checksums
o End-to-end Data Encryption and Checksums

Of course adding any kind of security will "impact" performance (Hi Rick - he hates that. ;)

However, if you're running this stuff across a 100MB link you're probably not too overly concerned about performance anyway.

Dave

[rick jones]

Tsk tsk tsk :)

Wisdom Teeth are impacted, people are affected by the effects of events.

Now, enabling security - especially encryption - will _affect_ :) service demand (quantity of CPU consumed per unit of work - something netperf will report for its tests if you ask it to), and a sufficiently large increase in service demand may _affect_ throughput.

How much is, of course, one of those (in)famous "it depends" sorts of questions.

[Gary L]

Thanks Dave for your good explanations above.

Those two HP-UX servers are running 11i v1 (although we do have v2 and v3 OS installation CD/DVD for so many years, but so far my boss no upgrade plan), I haven't heard Secure NFS in v1, It look like it using SSH protocol.

In this case, I think transfer speed is more important than security.

-G

[OldSchool]

Hmmm.. American Heritage sayeth thusly -

The effect or impression of one thing on another:
"..still gauging the impact of automation on the lives.."

[Olivier Masse]

I'm assuming you're transferring rman files or exports. You might actually be able to save time with a compression-enabled SFTP if the data can achieve a good compression rate (especially if it's exports) but this will take considerable CPU time. Forget it if it's already compressed, as you'll need a too much CPU cycles to (re)compress and encrypt all that data.

Good luck

[Bill Hassell]

Another alternative for maximum throughput is to use APA (called teaming on other platforms) where you use 4 LAN cables working together as a single IP address. Note that APA for throughput requires a minimum of 4 separate data streams (ie, 4 different copies of rsync, or sftp). Note that if you run 4ea Gbit links between two machines, you can easily push the limit on disk speeds (150-350 Mbytes/sec). That means that your 150 GB worth of files can be transferred in about 7 to 15 minutes, assuming that there are several files transferring at the same time. Now that speed is based on plain old FTP (no encryption, jumbo frames on simple crossover cables using Gbit cards on APA and fibre disks on both sides. The rate drops to less than half of that when using rsync, especially when there are lots of smaller files (a few MB each).

[Steven Schweda]

> [...] which way (speed/rate)is faster?

If I _really_ wanted to know, rather than
asking people who know less than enough about
my whole environment, I'd run a test. (I
know that it's a radical concept, but I'm odd
that way.)

> tar cvf - filename | remsh hostname "cd /dir ; tar xvf -"

If your CPUs are faster than your network,
adding compression (gzip, bzip2, ...) to that
pipeline might help. Full command examples
abound in this forum.

>.Wisdom Teeth are impacted, people are
> affected by the effects of events.

Hmmm. I'd say that people are affected by
events, not, generally, by the effects of
events. (There may be a limit to the amount
of wisdom which can be crammed into one pithy
remark.)

> American Heritage sayeth thusly -

Sadly, even good dictionaries tend to include
defective usage when it becomes sufficiently
common. Diluting "impact" to mean no more
than "affect" (or "effect") destroys much of
its value. (Dare I say, its _impact_?) One
(perhaps the only) benefit of using "impact"
everywhere is that semi-literates who haven't
mastered the difference between "affect" and
"effect" seem to be able to cope with
"impact" and "impact".

[Gary L]

Thanks Olivier, Bill and Steven for your details explanations and suggestons.

-Gary