VMS V7.3-2 Crashes Right After Upgrade

[Andreas Freiherr]

Hello Experts,

After upgrading from V7.2-1 to V7.3-2, during the first boot (the AUTOGEN run that completes the upgrade), an AlphaServer 1000A crashes immediately after the initial copyright message with a BUGCHECK, "code = 000001CC: INVEXCEPTN, Exception while above ASTDEL" and "Current Process = SWAPPER". After dumping memory, the machine tries to reboot and fails with the same problem again.

If I do a conversational bootstrap and USE DEFAULT, the system boots and runs (very slowly), but as soon as I let AUTOGEN set new parameters, the BUGCHECK is back.

What can I do to identify the parameter or subset of parameters that causes the new system to fail?

Thanks in advance for any hints!

Regards,
Andreas

[Ian Miller.]

A suggestion - comment out all but essential entries in MODPARAMS e.g leave SCSNODE, SCSSYSTEMID. Do autogen with setparams but don't reboot. Compare SYS$SYSTEM:SETPARAMS.DAT with previous version from AUTOGEN before edit. This may give you clues.

[Martin P.J. Zinser]

Hello Andreas,

how much memory is installed in this box and how are page and swapfile setup?

Greetings, Martin

[David Harrold · Accepted solution]

Hi Andreas,

Yep, I ran across this issue when I did the upgrade. It has to do with the settings for PQL_DWSQUOTA and PQL_DWSDEFAULT being near the page boundary of an alpha memory page. If the values are at or below multiples of the page size, then you will see this issue. The rule of thumb we worked out was N*PAGESIZE+32 for those values.

So for instance, our settings for WSDEFAULT and WSQUOTA were 65535. This worked out to be 65535/8192 or 7.99. The correct number to work around this problem would then be 8*8192+32 or 65568.

I haven't found this documented anywhere yet or heard what ECO (or when) will fix this issue.

Hope this helps,

Dave Harrold

Hope that helps.

[Andreas Freiherr]

Thanks for your replies!

Ian, you supplied the right idea to pin down the offending parameter, and I was just close to the PQLs when I read the message from David, which provided a shortcut to the final solution.

Martin, to answer your question: the machine is stuffed with 640MB of memory, I have no swapfile and a 2 mio. blocks pagefile in SYS$SYSTEM. However, since David's formulas solved the problem, I'd say this isn't critical.

Now if somebody can even answer David's question about what ECO fixes the SWAPPER, that would make my first post in this forum THE perfect experience.

Thanks again to all, special kudos go to David!

[Uwe Zessin]

Dave, how did you diagnose _that_ one??

[David Harrold]

Hi Uwe,

Actually, I got lucky. When I called the support center, the cubicle mate of the person I talked to had just found the problem and figured it out. I guess I was the second one to find. Pays to be lucky, I guess... :-)

Dave

[Uwe Zessin]

Right. I don't remember where I read this:

"no amout of planning will ever replace dumb luck"

I can assure you from my own experience that this is very true!

[Ian Miller.]

As far as I can tell the fix is known for this but has not appeared in a released patch kit. It apparently is to do with the change to move process working set into P2 space.

[Jan van den Ende]

Andreas, Dave,

please do a little reply on the stream I opened for this.

I am very grateful that you prevented us from falling for this one, and I just what to honour that!.

Jan

[Guy Peleg]

As mentioned above this problem inroduced
by the move of working sets to S2 space.

We have a new SYSBOOT.EXE that fix the
problem. If you have a service contract,
please contact your local support center to
obtain the new image.

The soon to be released VMS732_SYS-V0300
ECO kit will have the fix in it.

Regards,

Guy Peleg
OpenVMS Engineering

[Guy Peleg]

One more thing.....

The critical parameter during the boot is
PQL_WSDEFAULT.

booting a system with SYSGEN parameter
PQL_DWSDEFAULT set to 16384 or 32768 (or a
value slightly less than these and rounded
up during SYSBOOT initialization) can cause
the system to crash.

If the working set list parameters are near
or at the 8192 byte boundary, raising the
SYSGEN parameter just 1 beyond that boundary
should allow the system to boot normally
(ie, 16385 and 32769), providing a
workaround.

Again, VMS732_SYS-V0300 will fix this.

Guy Peleg
OpenVMS Engineering

[Andreas Freiherr]

Thanks, Guy!

Your answer completes the solution. I'll download and install the ECO when available and then remove the workaround from MODPARAMS.DAT.

Perfect, thanks again!
Andreas

[Doug Phillips]

Dear Guy Peleg,

You referred me back to here from the thread I started:

http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=575702

but you haven't answered my question.

If you answer my question in that thread you can get the as yet unclaimed 10 points. So far, noone has explained this satisfactorily.

Here's some more background info:

2 weeks ago my partner (who was unaware of this problem) upgraded a 7.2-1 DS10 to 7.3-2. Its PQL_DWSDEAULT was set to 2032, which is on an 8192 byte page boundary (2032/16=127 pages). It did not crash on reboot.

We have been given two examples of values that will cause a crash: 16384 (1024 pages) and 32768 (2048 pages). We do not work with any system where it would be reasonable to set values this high.

We have discovered one example of a value that did not cause a crash: 2032 (127 pages). Most systems need more than this.

According to your formula, we should have set it in MODPARAMS to 2033, but we did not. So, why did it not crash?

Since all of the Alphas we need to upgrade are different, and all are production systems, and upgrading to 7.3-2 with all of the patches** takes a__very__long__time, putting arbitrary numbers into MODPARAMS without understanding the problem is at best just plain dumb.

Since the upcoming patch is for 7.3-2, it won't help this problem while upgrading to 7.3-2, just thereafter.

What is the real formula?

Doug

**Just keeping track of the stream of patches and recalls and supercedures is nearly a full time job.

[Guy Peleg]

Hi Doug,

I apologies for the late response I was swamped with other work.....

I also appologies for not being clear enough in my previous update.....

The problem occurs when the value of the
PQL_WSDEFAULT parameter is on a memory page boundry.

On Alpha - 1 page = 8192 bytes (16 BLOCKS)
On VAX - 1 page = 512 bytes (1 BLOCK)

The PQL SYSGEN parameters are measured in
pagelets (on Alpha)

1 Alpha page = 8192 bytes = 16 pagelets

When I mentioned that the error occurs on a page boundry I meant Alpha memory page boundry.

So when the size of the working set is
set to 8192 bytes or 8192 bytes * 2 (16384)
etc.... the system will crash.

Base on the above the following values
in pagelets will crash the system:

16,32,64,128,256,512,1024,2048,4096,8192,
16384,32768.....

Here is the calculation is reversed order...

If PQL_WSDEFAULT=32768 pagelets.
32768/16 = Exactly 2048 Alpha pages

32769/16 = 2048 + remain which uses another
page...which eliminates the problem.

I hope this is clearer. If you have more
specific question you may e-mail me offline.

Guy Peleg
OpenVMS Engineering
guy.peleg@hp.com

[Doug Phillips]

Thank you, Guy.

I'm glad you're keeping busy;-) I owe you 9 points. (10 points if you'd posted over in the other thread).

You don't have to reply, but since there are some other people I know who were just as puzzled as me, I had to keep pushing. Others can continue the discussion or ignore this as they wish.

The confusion was due to the use of the term "Memory Page Boundry". I learned that to mean that the memory [block] was a whole number of pages. It's a reference to such an address in memory, of course.

So, the crash prevention rule in pseudo-code:

Where n = a whole number > 2
and PQLv = PQL_DWDEFAULT value in pagelets

if PQLv .eq. 2^n then set PQLv = PQLv +1

Thanks again, Guy. That's enough for me.

Doug

[Doug Phillips]

Oops, head said 3, finger typed 2.

Correction:

Where n = a whole number > 3
and PQLv = PQL_DWDEFAULT value in pagelets

if PQLv .eq. 2^n then set PQLv = PQLv +1

[Guy Peleg]

Doug,

Memory pages.....blocks....bytes....pagelets
....vax memory pages....I must admit
that this is not trivial.

VMS tries to maintain compatibility
with previous versions/architectures of the
O/S. I guess sometimes it can turn to a
big spaghetti.

I'm glad this is clearer now, don't hasitate
to contact me in the future.

Guy Peleg
OpenVMS Engineering

[Doug Phillips]

Memory pages.....blocks....bytes....pagelets
....vax memory pages....I must admit
that this is not trivial.


Indeed, and that is why I'll persist a bit longer with this thread.

When a calculation is presented for something as important as "OpenVMS Crashes" it should use the correct terms and values.

The Memory Page Boundaries listed in the solution as causing a crash were:

16,32,64,128,256,512,1024,2048,4096,8192,
16384,32768,... pagelets.

However "Memory Page Boundary" describes **any** multiple of page size. Therefore, the pagelet values of 48, 80, 96, 112, 144, 160, 176, 192,... and any number times 16 are also memory page boundaries.

The problem is not if the parameter value falls on a page boundary, but if it falls on certain page boundaries. These boundaries are the "power-of-two" boundaries. There is another term for these special page boundary but it escapes me at the moment.

Plus, all parameter calculations should be expressed in the unit-of-measure of the parameter, or at least, conversion between different units should be shown.

In computing PQL_DWSDEFAULT, 8192 was expressed as the page size. In this calculation, the page size should either be expressed as 16 or as (8192/512) because the value is in pagelets.

Now, excuse me. I must go and finish tying my trash bags into neat little bundles:-)

Doug