data migration

[SAM_24]

Hi,

I am doing data migration from one location to another distant location. The amount of data is 100s of GB. I might need to transfer full data once and five time refresh after. Which utility is the best and effective to do it?

rsync
rdist
rcp
Or any other commercial or free software

Help.

Thanks.

[Nick Wickens]

Hi Raj

Sounds like a lot of data to transfer - can you not handle the problem with tape backups & recoveries ?. On the occasion I have had to transfer large amounts of data I've used the following tar/remsh command but you are of course limited to the 2Gb limits per transfer -

tar cf - /tmp/dir | remsh host2 "(cd /;tar xf -)"

[A. Clay Stephenson]

Before any kind of sane answer can be given, you need to tell us what sort of network connection you have, what sort of tape and/or removable media you have, how many hundred's of GB.

It might even make sense to make copies of the data and ship them via hard disks.

I doubt that rdist or rsync make sense for this because the transfer times will be so great that the systems will never sync. Of course, if you are running ATM then things are different.

[harry d brown jr]

Raj,

I'd like to add to Nick and Clay's questions:

(1) What kind of data?

(2) One large file, a few medium sized files, a bunch of small files, a few of each?

(3) can the data be compressed?

(4) Is this data on an EMC symmetrix? What kind of disk is it on?

(5) What OS's are you running?

(6) what kind of server is the data on?

(7) what kind of server is the data going to?

live free or die
harry

[Sridhar Bhaskarla]

Hi Raj,

To add further on the comments made, "r" commands can be very unproductive over the data of hundreds of GBs. Morever, if the data is a "database", you cannot use those commands directly.

You might need to consider different mechanisms. Take backup, move the tapes restore. And if you need, you can take incremental backup later and restore.

Since they are distant from each other, I cannot think of you having Escon links for tools like SRDF or HORC.

-Sri