Using LIB$BBSSI and BBCCI for locking

[Robert Gezelter]

John,

I actually meant the opposite (e.g., code changes would be more complex than anticipated).

It may well be mentioning the obvious, but the changes on the "straightforward" list often have significant impact at low risk and even lower effort.

I presume that disk shadowing is in use. The version of OpenVMS has not been mentioned, but (I've lost track of mu notes as to which version of OpenVMS Shadowing supports/will support) adding a RAM disk as a member of a shadow set), but either a RAM disk member or a attached RAM disk (external zero latency storage device) may help.

Also, are all images pre-installed? This can be a significant simplifier of activations.

On the RMS side, what are the default extensions of the files that are being created? Also, do the files need to be created in the same directory, or only accessible via the same logical name? Directory scattering (e.g., such as is done by MX and others) is a useful trick to spread diectory activity around.

I am sure that some of the above have occurred to you. Certainly, one of the limitations of a public forum is the amount of information available to responders. Some of the comments are also intended to provide information for others who follow this thread, now and in the future.

- Bob Gezelter, http://www.rlgsc.com

[John McL]

Hi Bob,

We're on VMS 8.3, we use shadowing, we use RAM disk for some things, all images are installed (but not resident). I'll shortly be floating ideas about merging certain images that create reports or how batch processes might do more work rather than exist only briefly. We also need to review whether some of our major report generating programs can be made to run faster.

While I'll been responding here I've also been making changes to the code we run at the end of each process so that it doesn't write thousands of individual files per day but composite files, one per node per hour.

There's a lot we might do and investigations are at an early stage. Our target is to reduce the overheads of process creation, image activation and to improve IO performance. Addressing the most heavily used of our own locks was just a small part of a much bigger picture.

[Robert Gezelter]

John,

As always, the context become clearer as more details emerge.

The devil is always in the details. As noted previously, public fora (e.g., ITRC) are very useful, but not a substitute for actual consulting expertise.

In terms of basic science, I would not presume that the MPSYNCH et al are the result of the application's internal locks. Even if they are locks on behalf of the application, they may not be THOSE locks.

Consider the benefits of INSTALLing more than just the file. Caution is advised, but the payoffs can be VERY large if an image is activated many many times. Ditto, reorganizing applications images so that commonly used sub-components are used from permanently loaded, shareable, resident libraries (this eliminates the work of relocating the user infrastructure on each initiation; it adds up).

"Thousands of files per day" is not a lot. My email gateway (an MicroVAX 3100/38) easily processes many more emails per day, creating three files per message without a problem. If the file creation is a problem, it may be a symptom, not a cause. One possibility that often gets overlooked is directory extension. Directories must be contiguous, and thus if they are expanding relatively rapidly, cause significant overhead as they are re-allocated and migrated. They can be pre-allocated, which resolves that problem. There are also other RMS and file system optimizations that can be done.

Also consider whether some optimizations can be hurting. If a disk is in effect an output-only device, it may pay to disable caching, as there may be better uses for the memory (the file system caches and RMS buffering scheme will prevent many unneeded writes and reads).

I hope that the above is helpful.

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

No one here is suggesting blindly making changes.

Clustering may well be causing you some problems; the sharing of disks and resources isn't a panacea. There are cases where partitioning can help here, whether with the locks and lock resource names, or with adding and segmenting disk spindles or file sharing.

Investigate the whole environment. Baseline it, too. This is a systemic process, and probably one underway.

If you think bitlocks are the way to go here, then you've already prepared the baseline (for comparison with the results) and the instrumentation and go for it and make the changes.

Personally, I'd be looking at the whole environment, as part of figuring where I wanted to end up. Sometimes the (actual) low-hanging fruit in a big application can be quite surprising. Sometimes tossing faster boxes (the going prices of used AlphaServer ES45 boxes? around US$500 rx2600? around US$300) can be reasonable, while in other cases finding and removing a critical resource bottleneck can have a disproportionate payback.

That's a decent-sized source pool you're working with. Not the biggest around, but big enough to make this project and this review a rather larger project. And of the scale where tools such as DECset PCA and such can and will help; there's no DTrace and no Instruments around, though.

[John McL]

I am fed up with the attitude of people responding to this thread. I do not expect to have to write "War and Peace" to describe our environment, nor to tell more about how we use our systems than I would like to, when I ask a question to this forum.

Looking back I can't see that anyone bothered to ask whether this was part of a tuning exercise or whether locking was just being reviewed. Try it next time because it makes for a more civil discussion.

(In fairness I note that this is the first time I've experienced such haranguing from this forum. I do however hope it's the last.)

Points will be assigned according to whether you stuck to the subject or tried to push the thread into a different area, or demanded answers to questions that you didn't show were relevant.