Fastest way to copy data from NFS filesystem to local disk

[Vincent O'Neill]

Hello !
I am looking for the fastest way to copy data from an NFS filesystem to a local filesystem. The filesystem is 65GB. I am thinking of doing a multi stream remote 'dd' copy. Any other suggestions ?

Thanks

[H.Merijn Brand (procura · Accepted solution]

Try to find a way that does not use the NFS protocol

remsh or ssh (preferable without encryption) both are way faster than NFS. Even Samba outperforms NFS by factors.
Not that I oppose to NFS, I'm using it everywhere, but not for big datatransfers.

# cd /new/folder
# remsh remotehost cd /folder/to/copy \; tar cf - . | dd | tar xf -

The dd acts as a buffer, causing more reliable tar acting

Enjoy, Have FUN! H.Merijn

[Robert-Jan Goossens_1]

Hi Vincent,

Backup and restore via a tape, or if you have gigabit nics on both sides you could use a remote tar or cpio.

# cd /orig_fs
# find . | cpio -pcmud /copy_fs

Hope this helps a bit.
Robert-Jan

[Arunvijai_4]

Hi Vincent,

Probably, this link should help you

http://www.sunmanagers.org/archives/1999/1914.html

-Arun

[Tiziano Contorno _]

Parallel background cp.

for DIR in $(find /NFS_FS -type d) | some awk first, second dir | sort -u)
do
cp -pr $DIR DESTINATION &
done

or use fbackup frecover pipe:
/usr/sbin/fbackup -f - -0 -y -i $DIR | (cd DESTINATION_DIR && /usr/sbin/frecover -r -oyX -f -) &

Regards.

[Vincent O'Neill]

Thanks for your input. I will investigate remote copies and tape backups. Points assigned.

[Zinky]

Vincent,

As one who has actualy done this inter-HP-UX/Solaris systems or accross the two many many times, the following method worked best for me:

Re-mount the NFS using UDP
mount -o proto=udp nfsserver:/nfsshare /nfsmount

(Why UDP? -- a whole lot faster than TCP..)

Then either use cpio or GNUtar to transfer /nfsmount's contents to /localfs. Depending on how /nfsmount is populated -- you can do a single stream or multi-stream cpio or tar:

Examples:

tar cf - /nfsmount|(cd /localfs;tar xf -)

cd /nfsmount
find ./ -depth -print|cpio -pdvmu /localfs


Hope this helps...

If you've a 100BT link -- expect the task to be done in about 2-3 hours (depending of course on the content.. if the're zillions of smaller files -- longer...)


If GigE -- significantly less than a 100BT.

Tape will be faster IF you are using a fast tape drive.

[Tom Henning]

I know the thread is closed, but just have to add my 0.02 USD in.

Don't forget to /verify/ that the data has transfered correctly. This will probably take just as long to complete as the copy did in the first place.