I've operated multi-site disaster-tolerant clusters successfully with RECNXINTERVAL set at the default value of 20 seconds.
In my case, I was able to run with lower RECNXINTERVAL values because I had two completely separate and independent extended LANs connecting the systems at the two sites, so that a Spanning Tree reconfiguration, which seemed to typically take about 35-40 seconds in those days (which is greater than the default 20-second value for RECNXINTERVAL), would be unlikely to affect both LANs at once.
In the old days, there was a recommendation of 180 seconds for RECNXINTERVAL in disaster-tolerant clusters of the day because that was the time required for a GIGAswitch/FDDI (or one of its line cards) to reboot, as it would have to do after a firmware upgrade. (But even that figure became out-of-date, as with the latest firmware revisions, that time in practice actually increased to 210 seconds for the 4-port FDDI line card to reboot after a firmware upgrade.) Perhaps your figure came from a conservative doubling of this old recommendation after the old figure of 180 seconds proved insufficient at some point in the past.
As another poster pointed out, in addition to the original IEEE 802.1d Spanning Tree Protocol there is now the Rapid Spanning Tree Protocol, IEEE 802.1w, which aims for much shorter reconfiguration times -- on the order of seconds or less rather than 10s of seconds.
I highly recommend that anyone for whom cluster interconnect reliability is critical and where the LAN is used as a cluster interconnect implement LAVC$FAILURE_ANALYSIS. This feature has been in VMS since 6.0 and generates OPCOM messages whenever a piece of the cluster interconnect LAN breaks (or when it is repaired). The EDIT_LAVC.COM tool from the V6 Freeware directory [KP_CLUSTERTOOLS] will help you set this up with minimal effort. LAVC$FAILURE_ANALSYS is documented in the appendices of the OpenVMS Cluster Systems Manual, and I had an article on the topic in the OpenVMS Technical Journal, V2 - see
http://h71000.www7.hp.com/openvms/journal/v2/index.htmlOnce LAVC$FAILURE_ANALYSIS is in place, then you will have a written record in console output and in the OPERATOR.LOG file, with timestamps, of all LAN outages, and thus their durations. If you run with this enabled for a while and see that your expectation of a maximum outage length of 60 seconds is what you're really seeing in practice (and you might even consider inducing some of the likely failure types during off hours while you still have RECNXITNERVAL set to 360 to see how the network really behaves), then you could lower RECNXINTERVAL to a bit longer than your maximum outage times with relative safety.
I once worked with a stock exchange which needed to run with RECNXINTERVAL=10 seconds and we used this technique to identify a LAN outage problem which turned out to be lasting 11 seconds (and simultaneously across three supposedly-independent LANs) and thus causing much grief. Armed with the timestamped info, the proof was available with which to, uh, enlighten the understanding of the network folks.
