OpenVMS data replication

[Albatross]

Hi Folks,

We have inherited the following setup:

- AlphaServer DS10 / 600Mhz
- OpenVMS 7.3-1
- MSA1000 SCSI disk storage, FC SAN
- TCP networking

We want to replicate the data of two internal volumes to another DS10 with identical setup.

We have been looking into solutions as ASCI's RemoteSHADOW (http://www.advsyscon.com/products/rso/openvmsdescription.asp), or Volume Shadowing, but with two non-clustered systems.

Please, may you help us with some advice regarding this requirement? Our local HP Advisors are taking forever to give us a recommendation,

Thanks a lot,

[Hoff]

If you're looking at shared access and load-balancing and failover, then clustering with host-based volume shadowing is usually considered the best solution.

Unless there are factors here that have not yet been disclosed (budget, substantial distances, network bandwidth and reliability, etc), I'm mildly surprised HP is pointing to the ASCI package as a solution here.

HP has and offers various remote replication packages for the XP and EVA series array packages (as differentiated from clustering), though I've not seen anything similar offered with the Modular Storage Array 1000 controllers.

Depending on your particular requirements and network bandwidth, there may be other near-line options.

[Robert Gezelter]

Albatross,

If this is a one time operation, then it is not properly referred to as "data replicaton". Data replication is used to refer to real-time replication.

OpenVMS volume shadowing requires clustered systems. If all that is required is the ability to copy the data once, then it is possible to simply use BACKUP/IMAGE to create a saveset of each volume. This saveset can be written to the local system, or to the other system (or vice versa) using DECnet transparent file access.

If something more realtime and ongoing is desired, then more details of the application are relevant.

- Bob Gezelter, http://www.rlgsc.com

[Andy Bustamante]

Albatross,

As pointed out, a VMS cluster with host based volume shadowing is the solution for real time data replication.

You don't provide details on how current your data needs to be replicated, how far the two systems will be from each other and how much downtime can be tolerated. You can replicate date with a tape backup and restore using FedEx for bandwidth. On the other end of the scale, a cluster shadowing data between two sites. If HP isn't responding promptly, there are consultants to be found here.

You may also ask to be put in touch with a VMS Ambassador. Contact Sujatha Ramani or Susan Skonetski at HP com.

Andy

[Stanley F Quayle]

I routinely duplicate systems using DECnet.

Build a minimal system disk on the receiving machine. It'll need Alpha base and DECnet licenses. And make sure it has different DECnet and IP addresses than the current machine. You will need scratch space on the new system at least as large as one of the original system disks.

Then you do:

1. BACKUP/IMAGE old-disk -> new-scratch
2. BACKUP/IMAGE new-scratch new-disk
3. Delete new-scratch file
4. Repeat until done.

You'll have to shut down the queue manager and anything not related to the copy on the source system, else you risk the saveset in step 1 not being valid. And you'll have to make sure database systems such as RDB are shut down.

You might also limit the blocksize in step 1 to 2048 bytes -- I find that larger blocksizes result in savesets with "missing" chunks of data. You might also want to do a SET RMS to make the network buffering very small (contra-intuitive, I know).

If all you have is TCP/IP, you can do this using NFS instead. I back up my local VMS machines to a Linux system using NFS.

I have scripts to do the DECnet backup automatically, traversing the list of disks on the source system. I provide them to my CHARON-VAX and CHARON-Alpha customers [Shameless Plug Alert (tm) -- I am a CHARON reseller]. You probably don't need anything as fancy.

As others have mentioned, it would be handy to know how far apart these systems are. If they're in the same room, you could plug the new MSA into the existing system, boot from the VMS distribution CD, and just do a bunch of disk-to-disk copies. It would be fast...

[Ian Miller.]

I also see there is IPStor Software by FalconStor but I have no experience of that.

Where are you located?

[Robert Gezelter]

Stan,

With regards to your posting earlier in this thread, I have not seen any problems with save set block sizes. There is a difference in save set size caused by save set record blocking breakage, but it is only significant when moving save sets over low bandwidth links (e.g., modems using KERMIT).

With regards to network block sizes, all sizes work with remote file access, in my experience. The networking blocking factors must be consistent with other quotas, otherwise there will a resource issue.

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

Well, if we've here moved from a discussion of production distributed data replication options along to inexpensive and expedient and error-prone solutions, here's a solution involving a handful of lines of DCL acting as a DECnet-based BACKUP server process:

http://labs.hoffmanlabs.com/node/598

Various other solutions discussed earlier in this thread have higher and often far higher reliability than this solution. This DCL solution is among the cheapest available, and doesn't react all that well to errors.

[Albatross]

Thanks to everybody,

Sorry for not posting some relevant details before:

- We need at least hourly data replication, RDB datafiles are about 40GB, ~3% max. change in 24 hours.
- Downtime tolerated can be 4 hours max
- The two servers in the main site are in the same room, connected via 100Mbps TCP/IP
- The secondary site is connected via a 1Mbps IP link, 200Km.
- Main Site: In case of problems with the first DS10, we have the second server ready to be booted from the MSA storage with a dual-root bootdisk, or to act as a "donor" for parts transplant.
- Secondary Site: In case of problems with the main site... well we use the tapes and the DBA works his best to bring the service back.
- We have no DECnet.
- As you might infer from our setup, our budget is pretty limited :(
- For (bad) design reasons, our application can not be used with VMS Clusters, as our provider told us many times.
- And finally, we are in Quito-Ecuador (http://en.wikipedia.org/wiki/Quito)

We think that maybe a block-based, IP replication solution will be the best fit.

Thanks again for your insight,

SE

[Hoff]

With a one megabit link between sites (and a link that is probably also used for and critical to other inter-site tasks), I'd look to the math (for whether you can get the bits stuffed over the link in your window) and potentially to a courier-based data transfer; I'd use the window to replicate to disk or DVD, and would ship the media over the two hundred kicks.

I am here presuming you already have regular transfers between the sites, and courier-based networking has rotten latency but massive bandwidth. And it's cheap and effective.

If you can buffer on a local NAS or DAS storage widget, you can extend out your window by buffering onto a local brick and using any traffic routing priority features you might have on your switches/routers.

There are various (cheap) options for removable storage that can be used as an intermediate storage location and removable disk storage bricks, and particularly of you have a gigabit-class LAN.

Quito es bonito. Soy de Nueva Inglaterra. Y buena suerte.

[Robert Gezelter]

Albatross,

The additional information does clarify.

One of the most important questions is what is the actual data volume involved, and can it be compressed.

Before doing anything, take an audit log of one day's transactions and use ZIP (with maximum compression) to compress it. The result may be surprising. 3% of 40GB is admittedly, still 1.2GB, but compression can dramatically change that picture.

With regards to DECnet and clustering. DECnet is not needed, FTP can also be used to transfer the files. Admittedly, it is not as convenient as it could be.

The cluster statement probably needs clarification. I have encountered many applications that are alleged to be not compatible with clusters. In the cases where this is actually a valid technical question, a more correct version of the statement is: This application cannot be run on more than one node of a cluster at a time.

This leaves open the possibility of using OpenVMS clusters to implement a "fast failover" capability, since the disks can be made accessible from both systems at the mainsite.

- Bob Gezelter, http://www.rlgsc.com

[Albatross]

Thank you everybody,

We have planned some tests with our DBA, to extract our DB increments each 2 hours, compress them and script some FTP to our remote site as an interim solution. Next step will be try some remote IP replica solution, combined with Hoff's suggestion (massive bandwidth via nightly-sent tapes)

Your input is **vastly** appreciated, thanks a lot and take care.

SE