topic Re: Question on DR test using cluster backup to standalone system in Operating System - OpenVMS

Question on DR test using cluster backup to standalone system

Victor Mendham — Wed, 10 Aug 2005 13:36:47 GMT

How to make a cluster of one in a DR test. Or how to remove cluster.

Basically, I'm planning a DR test of my cluster, but the dr company will only give me a standalone server.

I would like to restore to the DR test server the production image and then boot and ensure that the node will boot up. I suspect it might get stuck waiting to join a cluster, with the message, waiting to join or form a cluster.

If I boot from CD, and go to DCL, can I modify any files or settings to allow me to boot my node, or would I need to build the node from scratch? If I run Autogen from cd, can I get it to modify the files, I have restored from tape? SYS4sysdevice would be cdrom right and my actual data would be on a hsz disk.

Re: Question on DR test using cluster backup to standalone system

Andy Bustamante — Wed, 10 Aug 2005 14:29:32 GMT

You have two approaches.

First boot from CD, mount restored system disk and use $ MC SYSGEN and >SYSGEN READ :[SYSx.SYSEXE]ALPHAVMSSYS.PAR modify VOTES and EXPECTED_VOTES to create a 1 node cluster or disable clustering (VAXCLUSTER=0). Then >SYSGEN WRITE :[SYSx.SYSEXE]ALPHAVMSSYS.PAR (x is your system root)

You can also do an conversational boot. At the console
>>> boot -flags x,1 (where x is your system root again).

You'll get a SYSBOOT> prompt where you can modify system parameters.

SET VOTES=n
SET EXPECTED_VOTES=n
(or SET VAXCLUSTER=0)
CONTINUE

Once you have this system up you'll want to modify modparams.dat and make sure autogen will create a bootable system.

Re: Question on DR test using cluster backup to standalone system

Victor Mendham — Wed, 10 Aug 2005 14:34:19 GMT

Thanks, I figured we could change in sysboot> the following.
set vaxcluster 0
set disk_quorum ""
set startup_p1 "min"
cont
Then once Booted up, need to modify modparams.dat and the run an autogen.

Any thoughts?

Re: Question on DR test using cluster backup to standalone system

Andy Bustamante — Wed, 10 Aug 2005 15:29:53 GMT

You're exactly correct.

You can also SET WRITESYSPARAMS 0 in SYSBOOT> to prevent saving any changes. You'll want to run update modparams.dat and run autogen immediately after the boot.

"There are 9 and sixty ways..."

Andy

Re: Question on DR test using cluster backup to standalone system

John Gillings — Wed, 10 Aug 2005 17:29:55 GMT

Victor,

There's no reason you can't run your test machine as a 1 node cluster. One advantage to this is your recovery could then proceed by adding nodes as it becomes possible.

Rather than having a whole set of SYSBOOT SET commands to be entered at a highly stressful time, work out the required set of parameters, and write them to a file, say "RECOVERY.PAR". Your recovery procedures manual then includes the steps:

1) Restore backup to disk (or, better, insert removable recovery disk into shelf...)

2) >>> B disk -FL 0,1
3) SYSBOOT> USE RECOVERY.PAR
SYSBOOT> CONTINUE

and off you go.

Alternatively you could have a system root configured as the recovery system.

For all DR planning, you need to focus on the recovery side. Work backwards to determine what you need to save, setup or configure in advance to make the recovery as simple and free of potential problems. For example, for backups, don't think "how do I back up this data?" think "how am I going to restore this data?".

Write your recovery plan on PAPER and place it in a metal box with a candle and a box of matches...

Re: Question on DR test using cluster backup to standalone system

Eberhard Wacker — Wed, 10 Aug 2005 18:50:08 GMT

Hi Victor, nothing new, only an addition:
if the hardware of the standalone DR server is the same as the production cluster node X and you have it "to run a.s.a.p." then only boot conversational the root of X
SYSBOOT> set vaxcluster 0
SYSBOOT> c
In this case you can forget all the other parameters and autogen and such things.
And in general there is no need to adjust disk_quorum, votes, expected_votes etc. if you turn off the cluster boot.
Cheers,
EW

Re: Question on DR test using cluster backup to standalone system

Willem Grooters — Thu, 11 Aug 2005 01:59:10 GMT

I'm not sure on this, so just the ideas:

* If your cluster configuration allows it, you could set up your DR machine as a one-node cluster. The only thing you need to check is that is will boot without other members.
* Consider creating another "node" - that is: [SYSn] directory - that you can use to boot your DR system. Configure that environment to fit your DR system, and all you ned to do is boot from that directory.

Re: Question on DR test using cluster backup to standalone system

Willem Grooters — Thu, 11 Aug 2005 01:59:14 GMT

Re: Question on DR test using cluster backup to standalone system

Robert Gezelter — Thu, 11 Aug 2005 05:57:12 GMT

Victor,

Willem managed to post this beat me to the buzzer on this one (ran out of battery last evening before I could post).

My strong preference is to pre-configure alternate bootstrap scenarios as different system roots, with all parameters pre-set. This avoids the error prone manual resetting of parameters under battle conditions.

For example, if my normal site used system roots:
0 - node ALFA
1 - node BRAVO
2 - node CHARLIE
3 - node DELTA

I would create an additional set of roots for the scenario of booting as a single node cluster for DR purposes:

10 - node ALFA
11 - node BETA
12 - node CHARLIE
13 - node DELTA

I would also place a safety check in the startup files, probably in SYS$COMMON:[SYSMGR]STARTUP_VMS.COM (or another file FAR earlier in the STARTUP database sequence of execution) that did a console inquiry and required a SPECIFIC RESPONSE (a keysequence not entered in a cavalier fashion) in the event that the system is booted from one of the DR/SingleMachine cluster roots.

The startup database is somewhat documented in the System Manager's documentation under the SYSMAN program. Similar techniques are mentioned in my presentation from the 1999 Fall DECUS Symposium, "SYSMAN for Improved Restart Performance" available from http://www.rlgsc.com/decus/usf99/sysman.pdf

Following this procedure, all that needs be entered in an emergency is:

- an alternate bootstrap code
- the response to the Verification Prompt described above

This reduces the potential both for accidental use of the emergency bootstrap, and errors trying to bring the system online at the DR site.

I hope that the above is helpful.

- Bob Gezelter, http://www.rlgsc.com

Re: Question on DR test using cluster backup to standalone system

Victor Mendham — Thu, 11 Aug 2005 09:15:45 GMT

Thanks all. This is something we will need to test and document in our DRP doc.

Re: Question on DR test using cluster backup to standalone system

Jan van den Ende — Thu, 11 Aug 2005 14:20:08 GMT

Victor,

one more point of interest, which may or may not apply to your specific situation.

Let us assume that the environment is "quite big" (metrics decided by the environment specifics)

Let us further assume there are multiple applications running.

If you are preparing a DR site, I must assume that at least one of those apps is pretty critical.

Now for the site-specific homework:
Can you pick some application(s) to be most critical, some to be non-critical but rather sensitive to downtime, and/or some for which downtime is a nuisance, but acceptable for a somewhat longer time?

If you answer "yes" to that, then it is worth considering some more complex planning (and maybe adapt your backup procedure to accommodate the following).
(This is an extension to John G's words: "work backwards from your DR plan")

Consider first get ONLY your critical app(s) running.
At first, forget about the rest, and take the most painful heat off as quickly as possible. That IS what critical means!
Only then restore the "just important" stuff, and start that.
And then, re-evaluate the pro's and con's for the less sensitive stuff. If there is any reasonable chance to revert to "normal" conditions, it might perhaps not even be desirable to activate those on your DR!

And...

if you really are serious about DR, think twice about the other meaning of DR: Disaster Resiliance. Multi-site clusters REALLY DO offer benefits! They might just be worth the inverstment.

hth.

Proost.

Have one on me.

jpe

Re: Question on DR test using cluster backup to standalone system

Robert Atkinson — Mon, 15 Aug 2005 04:07:56 GMT

Victor,
we have a similar scenario for our DR plan, but I avoided switching off clustering altogether for this reason.

Certain software checks the SYSGEN cluster values and makes descisions based on them. Switching this off may have an effect on the software.

In my opinion, you're better off setting the expected votes down to 1 in SYSBOOT.

Robert.

Re: Question on DR test using cluster backup to standalone system

Robert Atkinson — Wed, 01 Mar 2006 07:04:30 GMT

Victor - any chance you can assign some points to those that have helped you?

Rob.