Operating System - HP-UX
1748051 Members
4863 Online
108757 Solutions
New Discussion юеВ

Fastest file transfer between two HP-Unix sean rvers in same network

 
Mangal Pandey
Occasional Contributor

Fastest file transfer between two HP-Unix sean rvers in same network


I need to copy lot of small files between two HP box. I have an application which call the RCP scripts to copy these files but RCP is taking time and thus slowing the speed of application.
Can anybody suggest me any other file transfer techniques which will be the fastest one.
---Files are small (But there will be thousands of files at the same time to copy and a number of concurrent application will be invoked to copy number of files at the same time).
---Both servers are in same network.
---Need the fastest way to send file from one server to another.
14 REPLIES 14
Dennis Handly
Acclaimed Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

You may want to consider tarring up the files and if large, gzip, then rcp that tarball. Of course you would have to untar it on the other side.

I've seen others that said ftp was faster??
Steven Schweda
Honored Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

There's not enough information here to give a
reliable answer, and it's probably the wrong
question.

Why copy the files? Why not use NFS?

Why use "thousands of files" for any kind of
data sharing?

How often do the files change (and need to be
re-copied)? Who makes the files?

Why ask for the best way to implement a bad
solution to a problem?
Mangal Pandey
Occasional Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

Hi Steven,

There are two application running on two different servers. One application will generate the file and other has to process the files. First application generates all new files and in thousands of files per minute and second application has to process those files locally.

Can you suggest any approach for second application to handle the files faster assuming all time is taken in transferring files from on server to another.

you are telling something like NFS and i can implement this as well if this is the best way to do.
Steven Schweda
Honored Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

NFS would let a program on the second server
see (normally read/write) files which are
stored on the first server, without anyone
explicitly copying them.

I'd guess that NFS would be faster than using
"rcp" to copy the files, but an actual test
would be more reliable than my guess. It
would certainly be simpler. Some of the
speed depends on how the systems are
connected.

There are also various kinds of shared
storage available in the world. In the VMS
world, cluster members share disks with even
less effort than using NFS.
Emil Velez
Honored Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

I got a crazy idea.. If the boxes are HPUX 11.23 and you have ServiceGuard why not just use a cluster file system between the systems so the files are on both systems concurrently. CFS was released as a add-on to ServiceGuard and it uses Cluster VXVM and I have also used it for oracle data files with RAC but that is not needed if you just want a shared file system.

Jov
Honored Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

Hi,

I'd suggest either NFS or Samba/CIFS depending or what you need and comfortable in doing.

Go with NFS for its only and will likely stay with Unix access to the files, but if you think there might be a Windows element down the road, then setup Samba/CIFS on your 1st HP-UX server that is producing the files.

Have a read of the following post for details
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=3368

Jov
Mangal Pandey
Occasional Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

Based on the response above we have three choices NFS, Samba and CFS.

we can discard samba as no any windows interaction is there its all HP-UX servers.

Now for NFS, i read somewhere that it may affect the performance and sometime it is even slower then RCP. Is this true?

Now finally CFS is new to me and i'll definately google about it but a quick query here is that, will it affect the performance of application if read and write are done to CFS drives.

Thanks all for your responses.

Jov
Honored Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

NFS is your best and quicks way to setup. With CFS, you'll need an additional license where as NFS does not.


Jov
Steven Schweda
Honored Contributor

Re: Fastest file transfer between two HP-Unix sean rvers in same network

> [...] it may affect the performance [...]

Everything you do will affect the
performance.

I suspect that an actual test in your
situation will be much more reliable than any
estimates you'll be able to get here. You
already know how well rcp works. A test using
NFS should be pretty easy to arrange.