Data Replication

[The Brit]

We are VMS (7.3-2, fully patched) shop, looking for a Data Replication solution. Our sites are long distance (2400 Miles, ~50ms latency). The connection is an OC3, and the data rate is ~10GB/Day, peaking at less than 50% of the Pipe Capacity.
We are currently examining options from a couple of vendors but would be really interested to hear from anyone who is currently doing remote replication, regardless of the vendor or distance.
We currently use a combination of EVA's and an XP10000 for our storage, so we would be particularly interested in any information about solutions involving these systems.

thanks

Dave.

[EdgarZamora_1]

Hi Dave. I responded to this query in the Storage forum.

[Hein van den Heuvel]

Besides a description of the hardware, should you not also be describing the software involved to get reasonable help?

Is this an (Oracle) database application?
RMS (indexed) files? A file and print server?

IMHO that would be a major factor in understanding what the best solution might me, or what the limitations of any solution might be.

Regards,
Hein.

[The Brit]

We do not currently run any Oracle DB on OpenVMS, (but that might change).

The primary applicataion we are using is an in-house app (written in Dibol) and which uses mostly RMS files. We do not currently have them enabled for RMS Journaling, (although we would like to). From this perspective, we are (I am) trying to understand the implications and methodology that would be used to impliment Journaling in the existing setup.

While this does not really impact our ability to replicate, it certainly affects our recoverability in the event of a failure.

Dave

[Hoff]

The Encompasserve/DECUServe notes conference EISNER::VMS topic 3504 has a long discussion of data replication. FWIW.

[Colin Butcher]

Hello,

In general I find that doing the "data replication" in the application leads to the lowest amount of data to be copied around and it also gives you the best control over what happens when things don't quite work as expected. It's worked well so far - there are larg distributed systems out there with over 20 years of operation with zero loss of service that work this way. Because of limited (or exceedingly expensive) bandwidth we just had to figure out a reliable way of shipping data around over long distances. So, we ship the minimum we can get away with and protect the transactions as best we can so that we avoid the ultimate nightmare - which is corrupted data when you don't know it's been corrupted.

Don't expect to push all the work into the platform layers and have it just do everything for you in the most efficient manner possible. There are penalties for having things like controller based replication in terms of the amount of bandwidth you need and the worst latency you can tolerate. Sure, you save having to write a "better application" and save having to think hard about what exactly you need to replicate, but is it a good solution to your problem?

There are some handy things which can help:
- RTR
- NFS / DECdfs access to remote disc devices
- HBVS (probably not in this case as I guess this isn't a long-distance cluster because of the latency issues)

Connectivity and behaviour under failure may depend how you have the inter-site network configured (trunking between switches, EAPS ring, true dual LAN, etc.)

Things to consider from a design perspective are:

- synchronous v asynchronous
- what data matters, what doesn't matter so much or can get re-created
- what worst case latency can your applications tolerate if you're doing synchronous replication
- what worst case time lag can your applications tolerate in terms of your data being out of sync if you choose to go asynchronous

The main thing is to design for failure conditions rather than design assuming that it will all be working.

It's not easy. The "best" solution will depend on a lot of things, not least your organisations technical abilities and their willingness to do some hard work, plus how well you'll be able to support it all in the long term. No point doing something truly brilliant if it's a support nightmare. No point doing something simple if it doesn't actually work very well.

Have fun. Cheers, Colin (http://www.xdelta.co.uk).

[Robert Gezelter]

Dave,

I would pay particular attention to WHAT goes over the pipe.

I had a client site that had abysmal performance on their cluster, which was split between two sites with a dedicated fiber connection.

On review, I was able to determine that the system was configured to shadow all data, including the massive SORT/MERGE work files, across the link.

Moving the scratch files, which were not useful beyond the scope of a single process to disks that were not remotely shadowed was a major performance enhancement.

I try not to speculate on environments without sufficient data, but when shadowing volumes across distances, it is to your advantage to keep the volume small, either at the EVA level or by using a tool such as LD.

- Bob Gezelter, http://www.rlgsc.com

[The Brit]

We classify our data as either Tier 1, (critical), Tier 2 (Important), or Tier 3 (the rest).
Tier 1 is Host Shadowed with one unit in an XP, and one unit in an EVA8000. Tier 2 is also Host Shadowed, with both units in separate EVA8000's, and Tier 3 is unshadowed, and hosted in an EVA6000.
We do not attempt any remote shadowing (in the OpenVMS sense), it is totally out of the question with latencies ~50ms.
Our prime concern is that we have some level of data recovery available in the event of a Disaster. (sadly we are familiar with this having had a major flood in our datacenter several years ago).

Dave