Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

Upgrade VMS 7.3-2 to 8.3

 
SOLVED
Go to solution
Rich Hearn
Regular Advisor

Upgrade VMS 7.3-2 to 8.3

Hi Folks,

I'm going to be doing an upgrade of VMS on 2 ES47's in a cluster and I'm wondering if I can "get into trouble" by doing the following:

1 Get shadow copy of on-line cluster system disks
2 Move to DS20 test hardware
3 Remove all HSV disk & network connections from DS20 - no way to be seen on-line with live systems
4 Perform the upgrades specifying it will be in a cluster
5 Shutdown & do not re-boot for autogen until back on the ES47's the day of the upgrade (or should I do the autogens on the DS20's?)

Can anyone see if I'm setting myself up for problems? The goal is to minimize system level downtime (yet upgrade VMS) to allow for application upgrade also

Tnx for thoughts/ideas,
Rich
19 REPLIES 19
Volker Halle
Honored Contributor
Solution

Re: Upgrade VMS 7.3-2 to 8.3

Rich,

why not complete the upgrade on the DS20 ? You can still run @AUTOGEN on the ES47s after moving the disk back.

Think about what may be changing on the live ES47 systems after taking a snapshot of the system disk (password changes, any system-mgmt level changes, accounting, auditing, errlog, jobs in queues) - or do you have all those files on a disk other than the system disk ?

Also take into account possible changes to SYSUAF and RIGHTSLIST during the upgrade.

Volker.
Robert Gezelter
Honored Contributor

Re: Upgrade VMS 7.3-2 to 8.3

Rich,

Before I say anything else, in many cases a system upgrade such as you describe should be a painless process with no issues visible to users. Indeed, using a rolling upgrade (one node at a time), it should be possible to accomplish the transition with zero impact on users, just some careful system management of the user workload. However, as I will comment later in this post, planning for the unexpected is what separates the successes from the crises.

While it is not always feasible, I try to move SYSUAF, RIGHTSLIST, and related files to a common location that will not be replaced as part of the system disk change. Alternatively, you can point to the existing files on the current system disks and just bootstrap off of another "volume".

I would take this opportunity to create a REPRODUCIBLE way of cloning the system disk for the test configuration. It is an activity that is likely to be needed in the future, and it is a high payback activity. In particular, I would strongly consider using a separate system root for the 8.2 testing, so as to preserve the system root specific files unmodified. With some care, it is possible to then boot the production roots without having to do surgery on them twice.

Once testing is completed on the test system, I would copy the system pack to a local disk or storage array volume on the production side, and boot ONE of the production nodes from it. I would be postured (see my earlier comment about the cluster common files remaining where they are) for the possibility that a retreat may be required for unexpected reasons. OpenVMS is very good at not causing this problem, but I have seen sites blindsided by unanticipated applications issues, and the ability to "fall back" and come back later without damage is, as I said earlier, what makes the difference.

I hope that above is helpful. If I have been unclear, please let me know.

- Bob Gezelter, http://www.rlgsc.com
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3

Volker

Thanks for the thoughts on all the possibilities of changes after getting a copy of the system disk and doing the upgrade - very worth a 3rd & 4th look at what all changes - particularly the sysmgr acct.

Bob,

Oh to be able to do a rolling upgrade - alas, application (Cache - GE/IDX) does not allow - mandatory app chg with vms upg - not backward compatible. Cloning the sys disk is always reproducible - thanks to VMS & shadowing (gotta love it!) A "retreat" of VMS would be by "keeping" 1 of the 2 current system disks as a "backup" while the other sys disk becomes the "new" shadow member for v8.3. Booting one of the nodes isn't going to be a possibility unfortunately. Your post is clear and I thank you for your thoughts too.

Both of you give me some fresh "perspectives" to consider and I consider them well worth the time to think about.

Thank you again to both of you,
Rich


Jan van den Ende
Honored Contributor

Re: Upgrade VMS 7.3-2 to 8.3

Rich,

having been trough this many times, I can only concur with Volker and Bon.

Rolling upgrades are soooo nice!.

Not knowing Cache, I can only guess about that part, but we also had one app with similar issues.
You (temporarily) need some extra disk space, but can you not do a simultanious app "rolling upgrade"?
ie, do the OS upgrade "rolling". and the app as well.
Now, the node that uses the updated system disk also uses the updated app.
Clever use of logical names should enable that.

Of course, doing a sandbox (your DS20) exercise is indicated in a sensitive production environment.

Success!

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
Willem Grooters
Honored Contributor

Re: Upgrade VMS 7.3-2 to 8.3

If no data conversion or software adaption is required (other than the Cache runtime): complete the update on DS20, reboot it to join the cluster. In that case, you can do some testing before applying the updates to your live system.

To be completely safe, and you have the hardware and software environment (and time and propably budget), and your production environment is setup nicely (that is: as little references to physical disk as possible), you can think of taking an image copy of all disks that are touched in the upgrades, and update that environment, or update the original environment, either from the disks upadted on the DS20, or immediately from that ES47) and finsih installation (including AUTOGEN). Next, boot the other ES47 from that environment.
If something breaks, you can use the saved environment to continue processing and restore the original.

I concur with Bob that pulling all 'changeble' system files off the system disk (SYSUAF, RIGHTLIST, PROXY, ACCOUNTING, AUDIT....) is generally a good idea. certainly if you have multiple system disks (as will be if you have the DS20 pulled into the cluster).

WG
Willem Grooters
OpenVMS Developer & System Manager
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3


Jan

1st, Thank You for your thoughts. I appologize to all for not being more clear in my original posting - this is a 2 node VMS cluster (v7.3-2)... it is also a 2 node Application (Cache) Cluster. This is also made more "interesting" by the fact that the current version of the application (v5.0.20) does not function with v8.3 of VMS. That means an upgrade to the application as well (to v5.0.21). Because both nodes (both O/S & App pieces) work as a real cluster, the only way to upgrade the application is to shut it down - the 2 application versions (current & new) are incompatible with each other as well as with VMS versions. I will confess, I've never been "bored" at work :^)


Willem,

Thank You for your thoughts also. Data conversion is not required, but MUMPS recompiles are required (about 3 hours on an idle
ES47) - lots of routines. Because I'm using a copy of the "live" nodes in the test system (DS20), I can not boot the upgraded disk into the live cluster to test, but it would be nice if I could. I really don't want to go changing the SCSNODE & SCSSYSTEMID to allow that - there are enough things I can get wrong already :^)

The system disks are individual to each node, shadowed, and a copy of each current disk will be kept aside as backup - just in case of a problem. Going back will not be simple due to the fact that the application would also have to go back (another 3 hour re-compile) that would not make a lot of people happy.

I am in total agreeance with Bob's thoughts about the "changable" system files - I'll be taking steps to integrate them with the upgraded disks after they're booted.

Rich
Volker Halle
Honored Contributor

Re: Upgrade VMS 7.3-2 to 8.3

Rich,

adding a 3rd root (for the DS20) to the system disk(s) while doing the upgrade on the DS20 may not be a bad idea. Then you could boot the DS20 from it's own root and by just changing VAXCLUSTER from 0 to 2 or vice versa, you could bring that node into the cluster temporarily. While running V7.3-2 and V8.3 in the same cluster may not be 'supported for production', it should not create any problems. And why not compile the apps on the DS20 (if you have enough disks available), even if it may take a bit longer...

Volker.
Robert Gezelter
Honored Contributor

Re: Upgrade VMS 7.3-2 to 8.3

Rich,

A sidelight on the discussion.

If the disks are all at the same OS level, then switching to different roots on a common system disk is a far more efficient use of resources. Especially in a SAN environment, increasing members of mirror sets allows more efficiency in caching and other areas.

It would require a look, but recompiling the routines for 8.3 should not be a roadblock, even in the situation described. The more critical obstacle is the data. If the routines can be recompiled on the DS20, and then the common system pack can be updated, then it is possible to quickly fall back to the 7.3-2 system and applications if a problem is encountered.

If the data files need to be transitioned, it is a different story.

This approach requires admittedly more effort up front, but it is far more robust in the event of an unanticipated problem at a later stage in the process. I always counsel my clients with 24x7 systems that it is not the update process that is most critical, it is the planning for the potential, however small, that things will not go as planned, and a [temporary] fallback will be required to meet operational commitments.

- Bob Gezelter, http://www.rlgsc.com
Jim Geier_1
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3

Rich,

In July I upgraded a GE Healthcare (IDX) 3-ES45 cluster with a single shadowed system disk from OpenVMS 7.3-2 and Cache' 5.0.20 to 8.3 and 5.0.21 and was completely done in less than 2 hours. According to GE Healthcare and InterSystems, recompiling all generated routines was not necessary for the minor upgrade.

We do have a non-production cluster of an ES40 and a DS20e with 11 Cache' environments, so I had an opportunity to practice.

My biggest gotcha is remembering to clear the shadow flag on the system disk being upgraded. Seems I almost always forget that, even though it is well-documented. All other aspects of the upgrade went very smoothly and well. I also applied many patch kits to bring the systems to the most current patches since we only get a scheduled outage once every 3 or 4 months.

My biggest frustration is doing the upgrade in the middle of the night, when I'd rather be sleeping. My second biggest frustration is that InterSystems and GE Healthcare do not allow for rolling upgrades.

Suggestion: if your ES47 systems have local disks, do an image restore of the distribution CD to the hard disk -- booting from the distribution is so much faster that way. I keep a copy of the distribution CD on a local disk in all five of the systems for whatever future use and or need arises, like making a standalone backup of the system disk, or defragmenting the system disk, etc.
Robert_Boyd
Respected Contributor

Re: Upgrade VMS 7.3-2 to 8.3

I want to second Jim Geier's suggestion about building a copy of the distribution on a hard drive.

For my upgrades to V7.2-2, V7.3-2 and V8.2 and V8.3 I built bootable hard drives with the distribution media. I also loaded up all of the relevant patches for the system, TCP/IP, DECnet, and other layered products as well.

This shortens the actual upgrade process dramatically in some cases. If you have an environment where you can have the volume in a movable drive or on a SAN where you can present it to different systems as needed that can be a real win.
Master you were right about 1 thing -- the negotiations were SHORT!
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3


Volker,

Thanks for your response. Your thought of a 3rd root for the Test System is intriguing, but that would require more time than I currently have to be able to "play" with Cluster_Config.Com a few times on the Test system before I feel comfortable with doing it on the "live" system disk.
I'm looking to do the upgrade a week from this Sunday - Dec 9th. I have no doubt that if you're familiar with doing it, it could be done in short order. I, however, haven't done it in 15 years - I wouldn't feel comfortable taking that kind of chance on the future system disk without some practice. Doing the re-compile requires another copy of the DB (200GB) so the new version of the app can be brought up. Not able to do that - even ignoring the support contract concerns - none purchased for the test system.


Bob,

A common system disk is not our configuration... the 2 cluster members are 6 miles (physically) apart, so we have separate shadowed system disks for that reason. Yes, we could've put the roots on the HSV disks, but felt that it would be easier to diagnose if the node could, at least, be booted. We'd then have that to work with even if we lost the network connection to the HSV disks.

Regarding re-compiling the routines; they get stored in one of the DB's for the Cache system - it's not like re-compiling a different version of C code that I can store on my VMS disk - these are stored in the MUMPS environment. You have given me pause to the idea of using my current shadow copies - I'm thinking of re-shadowing the systems disks on the day before the upgrade (Sat the 8th) and do the upgrade on those - changed common files will be minimal. Thanks for that thought in particular. Also thanks for your time and other thoughts - do 'preciate


Jim,

Thanks for your thoughts on this. Interesting that you were told the routines didn't have to be re-compiled. I was told we had to. Originally, I was told the downtime for the upgrade with the re-compile would take the weekend - obviously, that's not the case. I was remembering the 3 hour time needed last year when we went from v5.0.5 to v5.0.20 so I'm figuring on 6 hours total downtime. If it's less, *no one* evers complains - but 2 minutes after the expected downtime passes with no system, the phone starts ringing - go figure... :^)

Glad to hear that you did a very similar upgrade as I'll be doing & it went well. I understand the "gotcha" you're talking about - my initial attempt with the actual test system disk also reminded me of the need to clear the shadow flag. Ahh, it's nice to know I'm still human... We start our upgrade at 05:00 - I will *not* interrupt our backups (00:00 - 05:00 if a failure occurs). I can well understand the frustration of not being able to do rolling upgrades - but since we get downtime once a year, I don't mind the total shutdown. I don't have any logical disks on our systems, but it sounds like it merits looking into. I will do that - for the future. I don't want to try to "squeeze" too much "new" in for me to do for the next week and a half.


Robert,

I appreciate your thoughts as well. Looks like I'll have to set up a disk on the SAN than can be used as a logical disk(s) for all kinds of items (VMS, Multinet, Cache)

Thank you all for your thoughts.
Rich
Dana Conroy
Advisor

Re: Upgrade VMS 7.3-2 to 8.3

I'm planning to upgrade our GE/IDX OpenVMS 7.3-2 cluster (running a non-clustered Cache 5.0.20 environment) to OpenVMS 8.3/Cache 5.0.21 in March 2008, so I've found this thread particularly helpful - thanks to all!

We are not yet taking advantage of Cache's clustering capabilities, but I am grateful that I'll be able to do a rolling upgrade.

Rich - An upgrade from Cache 5.0.5 to Cache 5.0.20 required compilation of all GE-authored Cache classes due to a significant change in Cache's class library that occurred around version 5.0.12. I'm not expecting another round of compiles for this upgrade.

Jim - Glad to hear (but not surprised) that your upgrade went well! No routine or class compiles were needed for your Cache 5.0.20 to 5.0.21 upgrade?

Dana
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3


Dana,

Thanks for the "memory jog"; I had forgotten about the ZLIB/ZCALL upgrade. I'll give GE/IDX another "ask" regarding the re-compile. It would be nice if it's not required.

Tnx agn
Rich
The Brit
Honored Contributor

Re: Upgrade VMS 7.3-2 to 8.3

Rich,
One other thing that you need to keep in mind, (while you ponder your options), is that you cannot upgrade a shadowed system disk! i.e. the system must be booted with SHADOW_SYS_DISK = 0. See section 4.8 of the upgrade manual.

This may impact some of the suggestions above.

Dave.
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3

Dave,

Thanks for the thought. I was reminded of that fact when I tried to do one of my test cluster nodes system disk - the upgrade process promptly informed me that you can't upgrade a shadowed system disk. Gotta love it when the system "reminds" you... oops :^)

Thanks,
Rich
Jim Geier_1
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3

Dana and Rich,
With our Cache' upgrade from 5.0.20 to 5.0.21, we were at IDXtend 9.0 with no GUI, so no routine or object compiles were needed. My understanding is that had we been at Flowcast 3.0 or 4.0 or using the GUI in 9.0, we would have had recompile objects, but not a full recompile of all routines.

Rich, I checked this with GE/IDX yesterday, so make sure you really have to recompile all routines and not just objects (I'm told there is a rather substantial difference in time required with routine compiles taking much more time).

We are now at 4.0, and starting the project to deploy the GUI, so my future Cache' upgrades will change.
Jim Geier_1
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3

Rich and Dana,
ZLIB/ZCALL -- my notes from GE for the upgrade say that as long as you are running the minimum version documented as compatible, the ZLIB does not need to be reinstalled. I was also told that the ZCALL does not need to be rebuilt - and that surprised me. We were running ZLIB 9.4 and ZCALL 4.7. I decided to recompile and relink the ZCALL image because of the change in operating system version, and found that the CZF.EXE was exactly the same size. Then I just copied the CZF.EXE from one bin directory to another. We have 11 environments on our two-node non-production cluster, so I performed the Cache' upgrade 22 times on the non-production cluster. I found later that I had missed copying the correct CZF.EXE file to a couple of the bin directories, and neither Cache' nor the applications seemed to care. But I would recommend rebuilding the CZF.EXE image anyway.

(We did have to make a change to the ZCALL for the application version 9.0 to 4.0 upgrade, but that was months after the Cache' upgrade, and a completely separate issue.)
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3


Jim,

Wow! You've been busy! Let me start with Thank You for all this information. I've got a question out to our IDX folks to see if the re-compiles are actually required.

We're not running the GUI at this point in time either, tho' we are on Flowcast 3.0 (it was optional for this version - tho' I'm told we *have* to get to v4.0 soon). Here's our version.txt info displayed by sylogin
VMS 7.3-2
MULTINET 5.1
CACHE v5.0.20
FLOWCAST 3.0
ZLIB 9.4
ZCALL 4.7
I appreciate you passing along the information about CZF.EXE - that's good to know. Ok, I've gotten the answer that they'll be doing the object re-compiles after the upgrade.

Thanks for all your efforts,
Rich
Rich Hearn
Regular Advisor

Re: Upgrade VMS 7.3-2 to 8.3


Just wanted to say "Thank You" to all of you who contributed to this thread. My upgrade went well (Sun 9-Dec-2007 from the VMS perspective) and I wanted to let you all know I'm gratefull for all your help.

Tnx agn,
RIch