Best size for HBMM Reset Threshold

[Toine_1]

Hello,

I have enabled HBMM on OpenVMS 8.3-1H1
I have set the HBMM Reset Threshold to 1.000.000

Every 10 minutes I have a bitmap reset.
Should I increase the Reset Threshold or is this value high enough?

$ sho shadow datadisk1

_DSA105: Volume Label: DATADISK1
Virtual Unit State: Steady State
Enhanced Shadowing Features in use:
Host-Based Minimerge (HBMM)

VU Timeout Value 64000 VU Site Value 1
Copy/Merge Priority 5000 Mini Merge Enabled
Recovery Delay Per Served Member 30
Merge Delay Factor 200 Delay Threshold 200

HBMM Policy
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 3 of the nodes: NVA,NVC,NVE,NVJ Multiuse: 0
HBMM bitmaps are active on NVJ,NVA,NVE
HBMM Reset Count 1533 Last Reset 14-AUG-2009 18:20:38.41
Modified blocks since last bitmap reset: 235458

Device $1$DGA105 Master Member
Read Cost 2 Site 1
Member Timeout 120

Device $1$DGA205
Read Cost 42 Site 2
Member Timeout 120

Thanks in advance

/Toine

[Volker Halle]

Toine,

this amounts to an average of 1666 blocks written per second. Do you think this is a 'normal' usage pattern for this disk ?

Section 6.8.2 'Setting the Bitmap RESET_THRESHOLD Value' in the OpenVMS V8.2 New Features and Documentation Overview manual discusses how to set the Reset Threshold Value.

Volker.

[Toine_1]

Hello,

Yes this is a normal behaviour of this disk.

Should I increase the Reset Threshold to have less bitmap resets to get a better performance.

Or just disable HBMM?

/Toine

[Volker Halle]

Toine,

if this IO-load is normal, you will probably pay some IO performance penalty every 10 minutes, so it might just be better to turn off HBMM on this disk or increase the Reset Threshold even more (to maybe 6,000,000).

Volker.

[The Brit]

Toine,
Can I ask you how far apart your shadow units are (physically). The "read Cost" settings that you have on your shadow members look to me like the pretty much restrict access to a single unit.

Also (and I know this is probably a stupid question), but have you considered spreading the IO between multiple disks.

Dave

[Toine_1]

Dave,

The distance between the two shadowset members is 1000 meter.

This disk on VMS is linked to one LUN on a EVA SAN. There the data is spreaded over 28 physical disks.

But it is perhaps better to have more smaller LUNs on OpenVMS when using Volume Shadowing with HBMM?

/Toine

[Robert Brooks_1 · Accepted solution]

You should use the largest reset threshold you can tolerate -- the larger the threshold, the longer the merge will take (because there are more blocks to merge).

When we wrote HBMM, we had no idea what sort of values to use as a default (and didn't have time to do the exhaustive quantitative testing we'd have preferred), so we used the very low value of 50,000. We upped that default to 1,000,000 pretty soon after the initial release once we saw how quickly merges completed, even on low-end storage systems.

There is a slight performance penalty incurred when the bitmap is reset -- a "quiet point" is needed (all writes are stopped, but reads continue).

A series of lock manager operations are triggered in order to reach the quiet point, and once all outstanding writes have completed, the bitmap is reset; write I/O's are then allowed. During the bitmap reset operation, all write I/O's are held in a queue within the shadowing driver.

If you have a very busy volume (as you appear to have) you may want to reduce the number of quiet points reached, and increasing the reset threshold is the way to do that.

You can experiment with reset threshold vs. merge time by using the /DEMAND_MERGE qualifier on $ SET SHADOW, although you may
want to be careful using that on a production system.

Note that a bitmap reset is not triggered automatically upon reaching the threshold; the number of dirty blocks is compared to the reset threshold every SHADOW_HBMM_RTC seconds.

Thanks for using HBMM!

-- Rob

[Toine_1]

Thank you Rob, Volker and Dave,

Now I know more details about HBMM.

I have set the HBMM Reset Threshold to 6000000 and this is working OK.

HBMM is a very good functionality within the shadowing software and is working very well.


/Toine