topic Re: Patch question in Operating System - OpenVMS

Patch question

Rich Hearn — Wed, 03 Mar 2010 18:49:54 GMT

Hi all,

OVMS83A_SYS-1000 has a "temporary fix" in place for an Intersystem's app problem stating that it will be removed when corrected - to quote:

"This is a temporary workaround for customers until the underlying problem can be corrected. Once this is done, this change will be removed."

How do I know when it's been removed? (I've found no other references to Intersystem in any of the .txt files in the 8.3 patch dir) Is this change still in OVMS83A_SYS-1600? How I know when it's been removed? Do I still need it? (I do run Intersystems Caché on my Alpha)

Thanks ,
Rich

Re: Patch question

John Gillings — Wed, 03 Mar 2010 21:15:43 GMT

Rich,

I'd suggest you ask Intersystems.

My interpretation of what you've posted is that OpenVMS has implemented a workaround to protect you against a problem in Intersystems software (interesting, as I don't recall ever seeing any fix like that in any previous OpenVMS patches). Once Intersystems has fixed their problem, and distributed the fixed software to customers (like you), some future OpenVMS patch will reverse whatever temporary fix they've implemented.

On the other hand, I'd guess that whatever they have done must be fairly benign, as the SYS patch will most likely be installed on far more systems without the Intersystems software than those that have it.

Re: Patch question

Rich Hearn — Thu, 04 Mar 2010 04:41:33 GMT

John,

Thanks for the response, tho'I'm still a bit unclear on this.

"Once Intersystems has fixed their problem, and distributed the fixed software to customers (like you), some future OpenVMS patch will reverse whatever temporary fix they've implemented."

How can I tell if that "future OpenVMS patch" has been issued or not? Does one assume the patch is is all versions of SYS until another SYS patch readme.txt *explicitly* states it's been removed?

Thanks for your thoughts,
Rich

Re: Patch question

Bob Blunt — Thu, 04 Mar 2010 05:11:17 GMT

The specific text of the release notes:

5.2.3 UNABLCREVA Bugcheck

5.2.3.1 Problem Description: When stopping and starting Intersystem's Cache database product, a fatal "UNABLCREVA, Unable to create virtual address space" bugcheck may occur. The change in this kit makes the UNABLCREVA bugcheck non-fatal. This is a temporary workaround for customers until the underlying problem can be corrected. Once this is done, this change will be removed.

Images Affected:

- [SYS$LDR]EXCEPTION.EXE

- [SYS$LDR]EXCEPTION.STB

- [SYS$LDR]EXCEPTION_MON.EXE

- [SYS$LDR]EXCEPTION_MON.STB

Image details for the EXCEPTION.EXE image from the VMS83A_SYS-V1000 kit:

o [SYS$LDR]EXCEPTION.EXE (new image)

Image Identification Information

image name: "EXCEPTION"
image file identification: "X-5"
image file build identification: "XBCA-0080070066"
link date/time: 29-OCT-2008 14:07:47.53
linker identification: "A13-03"
Overall Image Checksum: 534129896

$ ana/image/inter sys$ldr:exception.exe
to check for the image header information to check which file your system is running. The subsequent kit, VMS83A_SYS-V1100, has a different EXCEPTION.EXE but the latest VMS83A_UPDATE-Vxxxx kit info I have seems to still have the EXCEPTION.EXE listed here.

John's right, Intersystems should know if they've fixed their part. I hope that VMS Engingineering has been informed by Intersystems if it has. HOWEVER, having all the customers running the right Intersystems code in synch with the right OpenVMS/Alpha version AND the right EXCEPTION.EXE could be like trying to identify ball bearings blindfolded while wearing gloves. Put another way, who knows when all the cards will align.

Re: Patch question

Volker Halle — Thu, 04 Mar 2010 07:59:58 GMT

Rich,

a word of warning: NOTHING in this text states, that this a problem in the Cache code ! This is your interpretation of this text. It could also be an OpenVMS code problem - which is being triggered by cache database shutdown - for which an immediate solution could not be easily implemented in OpenVMS !

I have seen and reported this problem in MAY-2008 on OpenVMS I64 V8.3. HP had delivered a solution (patched image). The problem had happened again in SEP-2008 with V8.3-1H1 and all patches current at that time.

VMS831HI_SYS-V0300 had then been released on 3-DEC-2008 - with a 'workaround' to make this bugcheck non-fatal.

So you best ask HP about this.

Volker.

Re: Patch question

Volker Halle — Thu, 04 Mar 2010 10:04:06 GMT

Rich,

looks like you've found another patch regression !

VMS83A_SYS-V1000 contains EXCEPTION.EXE linked 29-OCT-2008. VMS83A_SYS-V1100 contains the same image - that's o.k. All the UPDATE patches also contain this image, that's fine, but...

But all the subsequent VMS83A_SYS-Vnn00 patches (-V1200 up to -V1600) do - according to the release notes - NOT contain EXCEPTION.EXE. That is NOT allowed, as all patches MUST BE CUMULATIVE !

There was another fix in EXCEPTION.EXE (case QXCM1000830697) in VMS83A_SYS-V1000, which now seems to be 'lost' as well !

Please log a call with HP !

Volker.

Re: Patch question

Volker Halle — Thu, 04 Mar 2010 10:12:33 GMT

Rich,

I just checked: VMS83A_SYS_V1600 does NOT contain EXCEPTION.EXE, so this IS A REGRESSION !

Volker.

Re: Patch question

H.Becker — Thu, 04 Mar 2010 10:31:11 GMT

As far as I know, this is an Integrity only problem. So the answer is: no, you do not need it to run Intersystems Cache on an Alpha.

Re: Patch question

Rich Hearn — Thu, 04 Mar 2010 15:39:35 GMT

Well, nothing like "stiring the pot" - eh?

Bob,

Thanks for the text of the exception. I had been through it which is what prompted my question. I've tried to check all the ovms83*.txt's for "UNABLCREVA","QXCM...","WFM...", "Intersystem" - no avail to finding anything *reliably* indicating it was/was not included in their respective patches. "EXCEPTION.EXE" & "X-5" also did not prove to be a reliable indicator either.

I don't have any EXCEPTION.EXE's anywhere on the systems, so I'm assuming the patch never found it's way to my systems.

Until John mentioned it, I never thought that Intersystems code could cause OVMS to Crash with a stop command - I always thought the Kernel "ring" would prevent that. If it did allow it, I thought that would be an OVMS fix regardless of Intersystems code.

Volker,

I'm running Cache 5.0.21 on a 2 node Alpha OVMS 8.3 (SYS v6.0) cluster (ES47's) - what version are/were you running on integrity when you saw the problem. The last time I had my systems "down" was for the time change for last fall.

Thank you for checking and finding the REGRESSION - I will be submitting a case after I finish here.

Mr Becker,

Unfortunately, I can't chance finding out "the hard way" that this *does* apply to more than just integrity servers. The IDX folks were the ones to tell me about this, but they didn't know if it was more than integrity servers either, so I need to find out so there's no problems on a "go live" date for an upgrade their doing.

Thanks though,
Rich

Re: Patch question

Volker Halle — Thu, 04 Mar 2010 16:36:10 GMT

Rich,

I had seen this crash on a customer system. I don't know what version of Cache they were running. An OpenVMS software crash is always a problem, that has to be escalated to HP. And so I did...

Believe me, you do have an EXCEPTION.EXE in SYS$LOADABLE_IMAGES: - it's a basic execlet of OpenVMS, the system won't boot very far without it ;-)

You can trust H.Becker.

You should still raise a call to HP about this 'mess'. And hope to get confirmation about the facts found out here.

Volker.

Re: Patch question

Rich Hearn — Thu, 04 Mar 2010 17:20:25 GMT

Volker,

Thanks for the info on where/how you saw it.
agreed on the escalation.

>Believe me, you do have an EXCEPTION.EXE...
Yes, I had only looked in [Sysroot.sys$ldr] - I didn't check SYS$COMMON:[SYS$LDR]. Ooops

>You can trust H.Becker.
my apologies to Mr Becker - noted.

>You should still raise a call to HP ...
Done

Thanks,
Rich
_

Re: Patch question

Rich Hearn — Thu, 04 Mar 2010 20:07:04 GMT

Volker,

Here's the "secret"...

Rich,

It is true the fix that you referenced is not in VMS83A_SYS-1600. That is because the fix was moved to the UPDATE series of ECOs and is found in UPDATE-V1000 and the most current version, UPDATE-V1200. This is one of the reasons that the UPDATE ECOs are listed as prerequisites for the SYS ECOs.

If youâ d like to check this yourself, follow the EXCEPTION.EXE image through the patches. The one from SYS-V1000 which has the fix has a link date and time of 29-OCT-2008 14:07:47.53

Thank you,

hp Technical Services
Consultant IV
GET/ES
NRSS

Re: Patch question

Volker Halle — Fri, 05 Mar 2010 06:55:44 GMT

Rich,

oh my goodness !!!

I now have to sincerely doubt, that the HP OpenVMS specialists still understand, how OpenVMS patches are supposed to work !

UPDATE patches are ALWAYS only a collection of previous CUMULATIVE individual patch kits.

Volker.

Re: Patch question

Ian Miller. — Fri, 05 Mar 2010 12:47:03 GMT

Myself and others in HP are well aware of this issue with patches and are doing our best to ensure this gets sorted out.

Do keep reporting these problems.

Re: Patch question

Rich Hearn — Fri, 05 Mar 2010 15:08:46 GMT

Volker, Ian,

Thank you both for your thoughts & endeavors - I'm glad we have to forums, but as we all know, it's not (yet :^) a perfect world. I'm just glad folks like you are "out there"

Tnx agn,
Rich

Re: Patch question

Ian Miller. — Fri, 05 Mar 2010 15:59:52 GMT

HP have analysed the contents of the various patches mentioned.

What was concluded from the analysis is that there is no issue with the SYS kits for V8.3 Alpha, and I64 and no images are missed and all the patches are indeed cumulative.

Re: Patch question

Volker Halle — Tue, 09 Mar 2010 07:39:22 GMT

Rich,

let me take the opportunity of this problem and try to summarize my view of this problem:

Doing OpenVMS support during the last 3 decades, I have come to the conclusion,
that everything being done by OpenVMS Engineering has been done for a reason.
While I don't remember ever seeing the OpenVMS patch strategy being publicly or
even internally documented, here is my perception of this strategy:

- OpenVMS patches are always cumulative. This rule applies to both individual facility patches (e.g. VMS83I_SYS-Vnn00) and the UPDATE patches. Each facility patch kit in itself must be cumulative.

- Once an image or procedure had been included in a facility patch kit, it needs to be included in all future versions of this facility patch kit, even if no further changes have been done to this image.

- The release notes of a patch describe the problems solved or functionality added by this specific version of the patch.

- Because of the cumulative nature of OpenVMS patches, all release notes of the previous patches for this facility (e.g. SYS or SHADOWING etc.) are included in each patch. This makes sure, that you get a description of ALL problems fixed in this patch (and it's predecessors) for this facility.

- OpenVMS UPDATE patches are just a consolidation of all current facility patches combined into one patch kit. This makes patching OpenVMS easier by having to just install one UPDATE kit instead of multiple facility patches.

- OpenVMS UPDATE are not allowed to contain any changes, that are not available as individual facility patch kits

- OpenVMS UPDATE patches include all bits and pieces of the most recent versions of all individual facility patches, including the release notes of all previous versions of those facility patches as described in the patch release note chapter 5 (example from VMS732_UPDATE-V1700):

Note that OpenVMS Patch kits are cumlative and that they contain all changes that have been released in earlier versions of the kit.
The issues addressed by these kits can be found in the following
Release Notes files after the VMS732_UPDATE-V1700 kit is installed:

o [SYSHLP]VMS732_ACC-V0100.RELEASE_NOTES

o [SYSHLP]VMS732_ACRTL-V0100.RELEASE_NOTES

o [SYSHLP]VMS732_ACRTL-V0200.RELEASE_NOTES
...

- OpenVMS UPDATE kits releases were being projected in the MASTER_ECO_LISTs since a couple of years. This allowed customers for planning ahead to reserve the required downtime for UPDATE patch installations.

- After starting with the 3-month release cycle for the UPDATE kits, new facility patches 'required' the current UPDATE as a pre-requisite, but this changed very soon based on customer feedback. Dependencies of new facility patches on previous UPDATE patches have been 'relaxed', whenever technically possible (Example: VMS732_AUDRSV-V0400 released 22-FEB-2007 did not require VMS732_UPDATE-V0900 released 7-DEC-2006, but could be installed against VMS732_UPDATE-V0800 released 14-SEP-2006). This feature had been implemented based on feedback by customers, allowing them to install urgent individual facility patches without having to install a bigger and newer UPDATE kit at the same time. OpenVMS customers are typically very serious about change management and stability of their systems.

- OpenVMS patches should never be re-released without incrementing the version number. Exceptions have been seen when non-critical kit contents has been changed (like correcting checksum or updating release notes). The reasons for those changes were usually being documented in the MASTER_ECO_LIST.

- In the past, a group of people in the CSC in Colorado Springs have been doing their own patch verification of the patches released by OpenVMS Engineering. This additional testing seems to have been abandoned.

- One of the early examples of this problem is the VMS831H1I_SYS-V0400 kit (released 9-JUL-2009), which is missing IO_ROUTINES present in VMS831H1I_SYS-V0300 (released 4-DEC-2008) and previous versions of the VMS831H1I_SYS kit. This violates the rule, that each individual facility patch kit in itself must be cumulative.

By requiring VMS831H1I_UPDATE-V0600 to be installed before VMS831H1I_SYS-V0400, the IO_ROUTINES from VMS831H1I_SYS-V0300 are being installed on the system, but this is not following the OpenVMS patch rule of 'cumulative' patches. This is some kind of 'shortcut' and not without risks. If all facility patches are cumulative in themselves and UPDATE patches are just a consolidation of facility patches, such a problem jsut cannot happen.

This list may not be complete and most likely does not cover all rules for the
OpenVMS patch strategy, but it is an attempt to summarize the most important
topics learned over time.

This is a very clean and straightforward design. Taking 'shortcuts', like with
the VMS83A_SYS-V1200 kit by taking images out of this kit and only providing
them in a required UPDATE kit, does not follow these rules and creates
unnecessary confusion and concerns within the OpenVMS community, as can clearly
be seen.

Volker.

Re: Patch question

Rich Hearn — Tue, 09 Mar 2010 19:48:24 GMT

Volker,

all I can say is Thank You.
Rich

Re: Patch question

Volker Halle — Wed, 10 Mar 2010 07:42:45 GMT

Here is a link to the OpenVMS Patch FAQ created by the previous OpenVMS patch team:

http://labs.hoffmanlabs.com/node/348

Volker.

Re: Patch question

Rich Hearn — Sat, 13 Mar 2010 14:35:32 GMT

Volker,

not ignoring your post, just a crazy week - thank you very much for the orig team patch faq - I suspect I'd not know to ask the "right" question to find it easily. Do 'preciate

Tnx agn,
Rich