topic Re: shadow cost in Operating System - OpenVMS

shadow cost

John Eerenberg — Mon, 17 Apr 2006 10:21:12 GMT

for a fibre channel san environment (using dark fibre) and an OpenVMS disaster tolerant cluster both seperated by roughly 10 miles, what values and when would I set the shadow sets for $ set shadow/read_cost. Assume seven 2member shadow sets.

Re: shadow cost

Robert Brooks_1 — Mon, 17 Apr 2006 10:47:32 GMT

The algorithm used by shadowing to determine which member to read is quite simple -- for each member determine the sum of the read cost and queue length. Read from whichever member has the lowest sum.

If you set the "remote" member to the highest possible read cost value, it'll only be read from if the "local" member is not available.

You can, of course, set a much lower read cost on the remote member and get similar behaviour if the queue length on the "local" member does not get too high.

You can the sample queue length for any device by using $GETDVI with the QLEN item code.

-- Rob

Re: shadow cost

Robert Gezelter — Mon, 17 Apr 2006 10:52:14 GMT

John,

A question and a thought. Is this a shadowset of mirrored volumes.

If it is not, it is well worth considering. If you lose the fiber link, you will also lose your protection against HDA failure. Remember, the corrolary to Murphy's law states that "HDAs only fail when the DR site is inaccessible".

- Bob Gezelter, http://www.rlgsc.com

Re: shadow cost

John Eerenberg — Mon, 17 Apr 2006 11:24:15 GMT

Each DGA device has a RAID1 behind the scenes. :)

So what are practical values if one wants to localize more then half of the read IO (I know it is app dependent)? Are there guidelines when to localize read IO?

Perhaps the following would work if I want to keep a read queue depth less then one local? Or do the default /read_costs already give me this?

DSA35 has $1$dga1 and $1$dga101.
Node A and EVA4000 A are in datacenter A with $1$dga1.
Node B and EVA4000 B are in datacenter B with $1$dga101.

NodeA$ Set Shadow/read_cost=2 $1$dga1
NodeA$ Set Shadow/read_cost=3 $1$dga101

On the other node
NodeB$ Set Shadow/read_cost=3 $1$dga1
NodeB$ Set Shadow/read_cost=2 $1$dga101

Re: shadow cost

Volker Halle — Mon, 17 Apr 2006 11:30:31 GMT

John,

the default read cost algorithm helps, if the system can identify which device is local vs. remote. In case of a FC SAN disk, this is not possible, so you might want to help shadowing with the SET DEVICE/SITE algorithm.

Volker.

Re: shadow cost

John Eerenberg — Mon, 17 Apr 2006 12:03:50 GMT

I guess I am trying to get a handle on whether or not the defaults are good enough *or* should I force keeping some/most of the read traffic local to a given datacenter.

Thoughts anyone?

Re: shadow cost

Jan van den Ende — Mon, 17 Apr 2006 13:23:17 GMT

John,

consider the speed of light trough glass: about 200.000 KM/sec, or 5 usec / km
Round trip = 20 KM = 0.1 msec extra latency (assuming the same components in the circuitry to local vs remote).
I very much doubt if that will be significant.
OTOH, IF extra switches or whatever are involved, find out THEIR latency, and calculate again.

hth

Proost (maybe in May in Nashua?).

jpe

Re: shadow cost

Thomas Ritter — Mon, 17 Apr 2006 18:16:53 GMT

John, why set any values ? We run a DT cluster which is completely symetric. Same work mix at both sites. Exact same configuration at both sites. Online sessions are exactly split, batch more or less.

Thomas

Re: shadow cost

Wim Van den Wyngaert — Tue, 18 Apr 2006 01:25:25 GMT

We define in sylogicals a logical defining the site (def/sys ING_SITE SITE1). Different for each site of course.

After the mount of the shadow set, we do a set device/site=f$tr("ing_site") for the shadow set (so, each building gets its own site for the complete shadow set).

Then for each disk of the shadow set, we do a set dev/site=x with x is being the site where the disk is located.

Normally I would expect no big performance gain unless you only have FDDI between the 2 sites without FC (FDDI is much slower than FC). We do ...

Wim

Re: shadow cost

Ian Miller. — Tue, 18 Apr 2006 04:54:43 GMT

Use the site settings as described by Jan and then VMS can tell which are local and which are remote and do the right thing. Reads will then mostly be from the local member disks at each sites.

See Host Base Volume Shadowing in action presentation at
http://www2.openvms.org/kparris/

Re: shadow cost

Thomas Ritter — Tue, 18 Apr 2006 18:10:44 GMT

Prehaps I'm the one not understanding something.We run a 4 node disaster tolerant cluster. Completely symmetric. Two nodes at each site connected to HSG80 SANS. The only disks we have local to a node are the system disks, pagefile disks and sortwork disks. Everthing else is on the SAN and accessed from both sites. All disks are shadowed and some are mirrored and shadowed.

Re: shadow cost

Volker Halle — Wed, 19 Apr 2006 00:52:39 GMT

Thomas,

when talking about 'local' disks in a SAN multi-site configuration in the context of OpenVMS shadowing for the calculation of READ_COST, we refer to the disks in the HSG/EVA in the SAME SITE as the OpenVMS system.

As all Fibre Channel disks appear 'local' to OpenVMS (as opposed to 'remote' MSCP-served disks), the shadowing software cannot determine, which disks are in the local site vs. the remote site and the SITE_ID algorithm allows the system manager to 'help' shadowing figure out the location (or distance to) the SAN disks.

The idea of the READ_COST setting is to prefer disks in the same site (i.e. typically with a lower distance across the SAN and therefore a lower latency) when reading from a multisite shadowset.

When you talk about 'local' disks in your cluster, you probably refer to SCSI disks in the box of your OpenVMS system.

Volker.