topic Re: INVEXCEPTN, Exception while above ASTDEL in Operating System - OpenVMS

INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Mon, 04 Apr 2022 10:26:58 GMT

Hello
Yesterday I had a crash of ES47.
Output of SDA>CLUE CRASH is attached.
Did anyone have simular problem?
Can it be problem with TCPIP?
OS version is 8.2 with newest patches (included TCPIP).
Regards.

Re: INVEXCEPTN, Exception while above ASTDEL

Volker Halle — Wed, 29 Nov 2006 05:36:56 GMT

Vladimir,

this is most likely a problem in TCPIP.

LDL R2,#X0030(R0)
BIS R31,#X01,R25
BEQ R2,#X00000B
LDQ R26,#X0008(R2)

R2 should have been loaded from #x30(R0) and contains an invalid S0 space address: A86A0A57 - access to #x08(r2) then fails with an ACCVIO.

To confirm this, have a look at

SDA> EXA 82430600+30
SDA> READ SYSDEF
SDA> FORMAT 82430600

This could also be generic pool corruption, but the output of the FORAMT command may tell more...

Volker.

Re: INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Wed, 29 Nov 2006 07:25:00 GMT

Thanks Valker
Here is the output:

Dump taken on 28-NOV-2006 14:55:21.42
INVEXCEPTN, Exception while above ASTDEL

SDA> EXA 82430600+30
FFFFFFFF.82430630: 00A5F685.A86A0A57 "W.jÂ¨.Ã¶Â¼."
SDA> read sysdef
SDA> format 82430600
FFFFFFFF.824305F4 WCB$L_PREVCOUNT 00000000
FFFFFFFF.824305F8 WCB$L_PREVLBN 00000000
FFFFFFFF.824305FC WCB$L_PREVRVN 00000000
FFFFFFFF.82430600 WCB$L_COUNT 827BF0D8
WCB$L_WLFL
FFFFFFFF.82430604 WCB$L_LBN 8229F780
WCB$L_WLBL
FFFFFFFF.82430608 WCB$L_RVN 61120060
WCB$W_SIZE
WCB$B_TYPE
WCB$B_ACCESS
WCB$C_MAP_PTR_LENGTH

FFFFFFFF.8243060C WCB$L_PID 001D00A8 TCPIP$A
CCESS_SHR+7E0A8
FFFFFFFF.82430610 WCB$L_REFCNT 00000000
FFFFFFFF.82430614 WCB$L_ORGUCB 81D7A8C0
FFFFFFFF.82430618 WCB$L_ACON 00000841
FFFFFFFF.8243061C 00000000
FFFFFFFF.82430620 WCB$L_NMAP 00000001
FFFFFFFF.82430624 WCB$L_FCB 827BF0C0
FFFFFFFF.82430628 WCB$L_RVT 00000000
FFFFFFFF.8243062C WCB$L_LINK 00000000
FFFFFFFF.82430630 WCB$Q_OPENTIME 00A5F685.A86A0A57
FFFFFFFF.82430638 WCB$L_READS 00000007
FFFFFFFF.8243063C WCB$L_WRITES 00000000
FFFFFFFF.82430640 WCB$L_SPLIT_IO 00000000
FFFFFFFF.82430644 WCB$L_ASSIST_IO 00000000
FFFFFFFF.82430648 00000000
FFFFFFFF.8243064C WCB$L_STVBN 00000001
WCB$C_LENGTH

FFFFFFFF.82430650 WCB$L_P1_COUNT 00000090
WCB$Q_DELIQ
WCB$W_P1_COUNT
FFFFFFFF.82430654 WCB$L_P1_LBN 048999E0
FFFFFFFF.82430658 WCB$L_P1_RVN 00000000
FFFFFFFF.8243065C WCB$L_P2_COUNT 00000000
FFFFFFFF.82430660 WCB$L_P2_LBN 00000000
FFFFFFFF.82430664 WCB$L_P2_RVN 00000000
SDA>

Re: INVEXCEPTN, Exception while above ASTDEL

Volker Halle — Wed, 29 Nov 2006 07:46:41 GMT

Vladimir,

the code has fetched the low-order longword of WCB$Q_OPENTIME(r2) - an OpenVMS 64-bit time quadword - and used it as an address...

SDA> EXA/TIME 82430630

should produce a valid time value.

The data structure pointed to be R2 could be some TCPIP-internal WCB. Try to pin it to some recognizeable TCPIP structure by using some pointers from this 'WCB':

SDA> VALI QUE/LIS 82430600 ! WCB$L_WLFL
SDA> SHOW PROC/ID=001D00A8 ! WCB$L_PID
SDA> FORMAT 81D7A8C0 ! WCB$L_ORGUCB
SDA> FORMAT 827BF0C0 ! WCB$L_FCB

Also try to verify pool using:

SDA> CLUE MEM/FREE
SDA> CLUE MEM/LOOK
SDA> SHOW POOL/SUMM/NONPAGED

Volker.

Re: INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Wed, 29 Nov 2006 08:08:11 GMT

Hello Volker
See attached file

Re: INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Wed, 29 Nov 2006 08:15:05 GMT

Sorry, forgot this:
SDA> EXA/TIME 82430630
28-NOV-2006 14:55:07.38
SDA>

Re: INVEXCEPTN, Exception while above ASTDEL

Volker Halle — Wed, 29 Nov 2006 08:41:25 GMT

Vladimir,

nonpaged pool and lookaside lists look o.k.

We will not be able to 'solve' this problem here. We can only collect evidence and document symptoms, so if anybody else is seeing this crash, they can confirm if it is the same problem.

Let me summarize the symptoms:

OpenVMS Alpha V8.2 TCPIP V5.5 ECO 1

INVEXCEPTN crash at IPL 8 with ACCVIO
at TCPIP$INTERNET_SERVICES+208AC
Failing instruction: LDQ R26,#X0008(R2)
R2 pointing to invalid address
R2 fetched from #X0030(R0)
R0 pointing to WCB

Here are some more things to look at:

SDA> SHOW DEV/ADDR=81D7A8C0 ! WCB$L_ORGUCB
SDA> READ SYSDEF
SDA> FORMAT 827BF0C0 ! WCB$L_FCB

SDA> FORMAT 827BF0D8
SDA> FORMAT 8229F780

The question still is, is this WCB the correct type of data structure the code is supposed to handle here. Using WCB$Q_OPENTIME as an address seems to indicate, that R0 should be pointing to something else.

Consider to escalate this to TCPIP engineering.

Volker.

Re: INVEXCEPTN, Exception while above ASTDEL

Volker Halle — Wed, 29 Nov 2006 08:45:26 GMT

Vladimir,

you can also use

SDA> TCPIP TAG ALL

to cause TCPIP to define SDA symbols for all of it's data structures.

Then try look at most of the data structures examined/formatted before with:

SDA> EXA address

Volker.

Re: INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Wed, 29 Nov 2006 09:59:00 GMT

Volker, thanks a lot for your time. I also did some examing and found no good explaination why server crashed. In hardware errorlog there is nothing indicating possible hardware failure.
Looks like a bug to me.
I escalated a call to HP support.

Re: INVEXCEPTN, Exception while above ASTDEL

Volker Halle — Wed, 29 Nov 2006 10:02:12 GMT

Vladimir,

once you receive a solution or explanation, please don't forget to update this thread.

Thanks,

Volker.

Re: INVEXCEPTN, Exception while above ASTDEL

John Travell — Thu, 30 Nov 2006 14:01:45 GMT

The code from TCPIP$INTERNET_SERVICES+20860: looks like the entry into a subroutine, and the cause of this crash as far as the evidence goes is that the calling routine supplied address FFFFFFFF.82430640 in R16.

The relevant instructions in the called routine...
TCPIP$INTERNET_SERVICES+20884: ADDL R31,R16,R1 copied this supposedly bogus value to R1, then
TCPIP$INTERNET_SERVICES+20890: SUBL R1,#X40,R0 does the next step.
TCPIP$INTERNET_SERVICES+208A0: LDL R2,#X0030(R0) loads what is actually the low l/w of WCB$Q_OPENTIME into R2, we don't know yet what it should have been. Finally,
TCPIP$INTERNET_SERVICES+208AC: LDQ R26,#X0008(R2) tries to access an unaligned address, and suffers the accvio...

The calling routine probably did a JSR R26,(R26) from FFFFFFFF.80669E3C.
You could do an ex/i 80669E3C-20;30 to check what is there.

We will NOT solve this without the listings for the relevant version of TCPIP, and getting access to those is like pulling hens teeth...
As Volker says, please (if you have not yet done so) log a call with HP, with the expectation of this being raised to engineering, and report the results back when they give you a solution.

Re: INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Wed, 06 Dec 2006 10:56:03 GMT

Hello
I got a response from HP support.
It says "This crash is caused by a known FTP related TCPIP 5.5 problem. The fix will be in the upcoming TCPIP 5.5 ECO2 kit."
They sent me attachment that contains 2 images to fix this problem.
So, looks like VMS support is doing the job well.

Re: INVEXCEPTN, Exception while above ASTDEL

Volker Halle — Wed, 06 Dec 2006 10:58:37 GMT

Vladimir,

thanks for reporting the solution. Which images did you get ?

Volker.

Re: INVEXCEPTN, Exception while above ASTDEL

STEPHEN CONWAY_1 — Wed, 06 Dec 2006 11:10:40 GMT

> getting access to those is like pulling hens teeth...

That's job security man :)

> So, looks like VMS support is doing the job well.

Your welcome.

> Which images did you get ?

It was a bug in TCPIP$INTERNET_SERVICES, and we always ship the ACP (TCPIP$INETACP) along with a new driver.

Note: TCP/IP V5.5-EC2 release is near. Will contain five fixes to NET.

Steve

Re: INVEXCEPTN, Exception while above ASTDEL

Vladimir Fabecic — Wed, 06 Dec 2006 11:19:05 GMT

Backup list shows tho images:
TCPIP$INETACP.EXE_ECO_E_A_V55ECO1;1
TCPIP$INTERNET_SERVICES.EXE_ECO_E_A_V55ECO1;1
So that means that TCPIP$INETACP.EXE and TCPIP$INTERNET_SERVICES.EXE should be replaced by next ECO.