Shadowset MERGE Throttling

[Jack Trachtman]

I'm about to enable merge throttling because from (unfortunately) real experience, the default settings cause the merge to impact user response time.

I've read about the throttling feature but don't have a good way of testing the settings (forced crashes are frowned upon here :-) )

Would sites that are using this feature be willing to answer these questions:

1) What values are you using for the logicals:

SHAD$MERGE_DELAY_FACTOR and/or
SHAD$MERGE_DELAY_FACTOR_DSAnnnn

SHAD$MERGE_DELAY_THRESHOLD and/or
SHAD$MERGE_DELAY_THRESHOLD_DSAnnnn

2) And (more importantly) how did you decide on these values?

Note: I'm separating this question from discussion of HBMM (HostBased MiniMerge),
which I will also be enabling).

Thanks all

[Uwe Zessin]

You might not need to crash a system.

In recent versions (I think you need at least V7.3-2) you can use:
$ set shadow /demand_merge DSAn:

[Robert Brooks_1]

In order to use SET SHADOW /DEMAND, one needs
either V8.2 (Alpha or I64), or V7.3-2 with any of the kits that contain support for HBMM. You do *not* need to enable HBMM in order to use
SET SHADOW /DEMAND, but the support for that command did not appear for V7.3-2 until
the HBMM V1 kit was released.

Note that the MERGE_DELAY logicals are not honoured for a host-based minimerge.


-- Rob

[Jack Trachtman]

Bob,

"Note that the MERGE_DELAY logicals are not honoured for a host-based minimerge"

Did I miss this in the Shadowing manual? Would you point this out to me in the doc? Thanks

[Robert Brooks_1]

I don't know if it's in the manual (and I'm too busy right now to check, but I will later).

In any event, an HBMM will happen so rapidly
(assuming reasonable RESET_THRESHOLD levels),
that there is no need to throttle it.

By "so rapidly", I'd expect most HBMMs to finish in a minute or two.

I do know that my statement (about ignoring the logicals) is correct, however; I verified that with guy here who knows shadowing from A to Z . . .

-- Rob

[Jack Trachtman]

Want to add a third question:

3) When the RESET_THRESHOLD value is reached/past and the block flushing is done
to allow bit-map clearing, has anyone noticed
any hits on response time while (in my understanding) all the nodes in the cluster pause to coordinate flushing and completing any write I/Os in progress?

[Uwe Zessin]

My memory might be faulty, but I thought I had used that command on a V7.3-2 field test version.

[Anton van Ruitenbeek]

Jack,

We only have set the logical SHAD$MERGE_DELAY_FACTOR to 100000 (max) in the LNM$SYSCLUSTER_TABLE.
The major reason is this: We have a multisite cluster, if there is a problem with a disk and this one is merging, in our situation this can take more then 15 hours. If during this period a problem on the other drive is hapening 'I GOT A BIG PROBLEM'. So the major reason is 'reduce the SPOF time'.
The users will notice this problem, but its in their benefit if the SPOF time is reduced.

Whe are also looking at HBMM, but cannot test it now so unfortuanaly this is keeping up. But it is always fine to have something to look for if you want it.

AvR

[Wim Van den Wyngaert · Accepted solution]

We have the delay factor on 400 but don't have a record why.

Are you sure that the performance penalty is caused by the merge itself ? All programs have to read at least 2 disks while in a merge, so the impact could caused by that too.

I already asked a solution for slowing the shadow copy. The only solution (I) found was suspending shadow_server for x seconds, than releasing it again for y seconds. And this in a loop. But this won't work in future versions.

Wim

[Wim Van den Wyngaert]

Check http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=707709