topic Re: Volume shadowing question in Operating System - OpenVMS

Volume shadowing question

Lokesh_2 — Wed, 05 Nov 2003 11:51:29 GMT

Hi ,

I have a query regarding volume shadowing on VMS. Consider a scnerio where a shadow disk DSA1 is having two shadow member named $1$DUA1 & $1$DUA2 . Now suppose I dismount one shadow member $1$DUA2 from the shadow set and after two days, I reboot the system ( Assume that during these two day some new data is written on shadow disk ) . What will happen after the reboot , if my system startup file contains the following line:

1)$ mount/system DSA0: /shadow=($1$dua1,$1$dua2) label

2) mount/system DSA0: /shadow=($1$dua2,$1$dua1) label

Can I assume my new data will not be lost in either case ?

Also please clarify me about the difference between shadow copying and shadow merging.

Thanks & regards,
Lokesh

Re: Volume shadowing question

Åge Rønning — Wed, 05 Nov 2003 13:47:20 GMT

Lokesh,
You can trust OpenVMS shadowing will ensure your data will be intact and that the copy operation will be done from the correct source in both example 1&2.
If you add a /CONFIRM you will be prompted after shadowing has decided which way to do it, But you probably don't want to have /confirm in your startup file...

Short version on how it works:
The mechanism used at shadow set mount time to make this determination, the Shadow Set Generation Algorithm, is based on four pieces of informationstored on each disk volume:
- Volume label ("should" be the same)
- Volume correctly dismounted
- Backup revision number
- Shadow set generation number
The shadow set generation number represents the time and event that caused the shadow set to change state. Such events are:
Addition of a volume
Removal of a volume due to HW error
Removal of volume by buser request
When entire shadowset is dismounted.

I'll look for more details or a pointer to a presentation I will post to prevent to much typing, But you've probably got the basic idea why VMS are able to handle your shadowset correctly.

Shadow merge is used when shadowing sw determines that the members of a shadow set are equally valid, BUT inconsistent. They may differ by a few incomplete write request after a system crash or similar.

Re: Volume shadowing question

Åge Rønning — Wed, 05 Nov 2003 14:09:49 GMT

You could find more about Copy/Merge in the OpenVMS FAQ at http://h71000.www7.hp.com/wizard/faq/vmsfaq_021.html#volshad

Re: Volume shadowing question

John Gillings — Wed, 05 Nov 2003 22:50:57 GMT

Ã ge is correct, shadowing will do its best to ensure that only the latest data is used to reform the shadow set.

However, there is one circumstance where you could lose data. That is if $1$DUA1 is unavailable at the time the MOUNT command is issued. Since $1$DUA2 is a valid disk, if Volume Shadowing software cannot see $1$DUA1, there is no way to know that there is a later copy of the data and the shadowset will be established using $1$DUA2. If $1$DUA1 is added later, it will be overwritten.

You can protect yourself against this scenario using the /POLICY qualifier on your MOUNT command. MOUNT/POLICY=REQUIRE_MEMBERS specifies that all proposed members be available before the shadowset is formed. The MOUNT command will fail if any members are missing.

You may also wish to use /POLICY=MINICOPY. MINICOPY means when a member is removed, Volume Shadowing software keeps track of changes to the disk. When the member is returned to the shadowset, only the changes need be copied.

That said, you should avoid reducing any shadowset below 2 member

Re: Volume shadowing question

Jack Trachtman — Thu, 06 Nov 2003 12:06:53 GMT

The replies to the original question have been good at summarizing some procedural issues with using Shadowing.

Can someone explain something I've never understood: Why is a "Copy" shadow operation really a type of Merge? At least from what I've read, adding a new member to an existing shadow would seem to require simply a straight copy from an existing member to the new disk, but I've seen references saying that this scenario is too simple and a type of merge is actually done in this case.

Re: Volume shadowing question

Keith Parris — Thu, 06 Nov 2003 20:44:23 GMT

The I/O algorithms for Copy and Merge are indeed similar to a large degree. For details and the rationale behind them, see Scott Davis' Digital Technical Journal at http://www.research.compaq.com/wrl/DECarchives/DTJ/DTJ301/DTJ301SC.TXT

Basically, Shadowing has to present the same semantics as a JBOD to the world:
1) Reading the same blocks should always return the same data
2) When something is written to a set of blocks, once the write is signalled as completed, that new data is returned to all subsequent requests for that block range; in other words, older data is never allowed to be returned after newer data has been returned once.

Shadowing has to meet these requirements despite the fact that there are actually 2 or 3 disks present, and that for performance reasons we are normally allowed to read from any one of the members (since they're always supposed to be identical).

So special care is required to handle cases like simultaneous reads and writes from different nodes in the cluster even while a Copy or Merge is going on (and even the possibility of a shadowed system disk where a node that is not even in the cluster at the moment may be either booting, or writing a crash dump file, at the same time as nodes in the cluster are using the shadowset). This is why the algorithms tend to be a bit non-intuitive.

There's more info on the algorithms and their performance implications in my presentation on Volume Shadowing Merge & Copy Performance at http://www2.openvms.org/kparris/