topic Re: data migration in Operating System - HP-UX

data migration

SAM_24 — Mon, 19 Aug 2002 20:39:44 GMT

Hi,

I am doing data migration from one location to another distant location. The amount of data is 100s of GB. I might need to transfer full data once and five time refresh after. Which utility is the best and effective to do it?

rsync
rdist
rcp
Or any other commercial or free software

Help.

Thanks.

Re: data migration

Nick Wickens — Mon, 19 Aug 2002 21:10:21 GMT

Hi Raj

Sounds like a lot of data to transfer - can you not handle the problem with tape backups & recoveries ?. On the occasion I have had to transfer large amounts of data I've used the following tar/remsh command but you are of course limited to the 2Gb limits per transfer -

tar cf - /tmp/dir | remsh host2 "(cd /;tar xf -)"

Re: data migration

A. Clay Stephenson — Mon, 19 Aug 2002 21:26:16 GMT

Before any kind of sane answer can be given, you need to tell us what sort of network connection you have, what sort of tape and/or removable media you have, how many hundred's of GB.

It might even make sense to make copies of the data and ship them via hard disks.

I doubt that rdist or rsync make sense for this because the transfer times will be so great that the systems will never sync. Of course, if you are running ATM then things are different.

Re: data migration

harry d brown jr — Mon, 19 Aug 2002 22:24:56 GMT

Raj,

I'd like to add to Nick and Clay's questions:

(1) What kind of data?

(2) One large file, a few medium sized files, a bunch of small files, a few of each?

(3) can the data be compressed?

(4) Is this data on an EMC symmetrix? What kind of disk is it on?

(5) What OS's are you running?

(6) what kind of server is the data on?

(7) what kind of server is the data going to?

live free or die
harry

Re: data migration

Sridhar Bhaskarla — Tue, 20 Aug 2002 01:13:43 GMT

Hi Raj,

To add further on the comments made, "r" commands can be very unproductive over the data of hundreds of GBs. Morever, if the data is a "database", you cannot use those commands directly.

You might need to consider different mechanisms. Take backup, move the tapes restore. And if you need, you can take incremental backup later and restore.

Since they are distant from each other, I cannot think of you having Escon links for tools like SRDF or HORC.

-Sri