CRASH ANALYSIS of VAX/VMS server

[Volker Halle]

This is the call frame of your application to SYS$PUTMSG:

SDA> sho call/next

Call Frame Information
----------------------
Call Frame Generated by CALLS Instruction

Condition Handler 7FF751D4 00000000
SP Align Bits = 00 7FF751D8 2FFC0000
Saved AP 7FF751DC 7FF75248
Saved FP 7FF751E0 7FF75224
Return PC 7FF751E4 005E8ED3 <
R2 7FF751E8 005D8E4C
R3 7FF751EC 0001827A
R4 7FF751F0 0000000F
R5 7FF751F4 7FF75298
R6 7FF751F8 005D8EE4
R7 7FF751FC 005D8E08
R8 7FF75200 005E8000
R9 7FF75204 00602484
R10 7FF75208 7FF75627
R11 7FF7520C 7FF7562D
Align Stack by 0 Bytes =>
Argument List 7FF75210 00000001
7FF75214 7FF75218

The MSGVEC argument starts at 7FF75218. Please examine:

SDA> EXA 7FF75218;50

Volker.

[Sk Noorul Hassan]

Volker,
Pls find the stack limit

[Sk Noorul Hassan]

Pls find the output of
SDA > exa 7FF75218;50

[Volker Halle]

The stack limits show, that the failing address in question is outside any of the user, supervisor, executive or kernel stacks.

But the MSGVEC argument to the SYS$PUTMSG call looks VERY suspicious:

The call frame to SYS$PUTMSG from your code (return address = 005E8ED3) presents ONE argument to SYS$PUTMSG - the argument list is on the stack:

Argument List: 7FF75210 00000001
______________ 7FF75214 7FF75218

The first argument to SYS$PUTMSG is the message vector MSGVEC passed by reference. So the MSGVEC starts at 7FF75218 and looks like this:

7FF75218: 7FF7562D <<< def msg options & arg count !
7FF7521C: 0001827A <<< message code = %RMS-E-EOF
7FF75220: 00000000 <<< STV value
7FF75224: 00000000
...

The argument count field, the low 16 bits of the first longword in the message vector is DEFINITELY WRONG ! It should have been 0001 ! EXE$PUTMSG 'walks' the argument vector based on the argument count. It may well walk into in-accessible address space based on that incorrect value of 562D.

Please use SDA> SHOW PROC/IMAGE to find out, which code is at address 005E8ED3, find the map and listings and the programmer (!) and check the construction of the MSGVEC for this call to SYS$PUTMSG.

Volker.

[Volker Halle]

Please also provide the data for:

SDA> EXA 19C250
SDA> EXA @19C250
SDA> EXA/INS @19C250;10

This could finally even lead us to determine the reason for the HALT

Volker.

[Volker Halle]

This a not a real valid OpenVMS HALT restart bugcheck, the APPLICATION itself is issueing BUGW BUG$_HALT bugchecks internally !!! This explains, why the registers and the HALT code (e.g. in the RPB) do NOT provide valid values.

Here is an example instruction stream from the application:

00195D27: BRW 00195DEF
00195D2A: PUSHL #10000002
00195D30: CALLS #01,@0019C250
00195D37: BLBS R0,00195D4D <<< return address
00195D3A: PUSHL R2
00195D3C: MOVL #0EC0DE07,R2
00195D43: BUGW #0160 <<< issue HALT bugcheck !!!

The saved PC on the kernel stack should point to the application code declaring the HALT bugcheck, so please provide the following data:

SDA> EXA/INS 195FEB-10;10

The non-fatal bugcheck entries can be reproduced easily with the following piece of MACRO code:

.entry halt, ^m<>
$cmkrnl_s routin=halt_cpu
MOVZWL #1, R0
ret

.entry halt_cpu, ^m<>
bug$_halt=^x160
BUGW
.WORD ; issue a HALT bugcheck
ret

.end halt

Fix the problems in your application and those crashes will go away. If you need further assistance, please consider to contact me offline (see e-mail address in my ITRC profile).

Volker.

[Sk Noorul Hassan]

Volker, Sorry for the late response. Pls find the attached SDA output as asked by u.

[Volker Halle]

Now what code is at 000BE3F4

SDA> EXA/INS 000BE3F4;20

I'm even more interested in:

SDA> EXA/INS 195FEB-10;20

This PC is found on the kernel stack in the crash and should reflect the last instruction executed before the crash !

Volker.

[Volker Halle]

continued and fully explained in:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=998774

Volker.

PS: Please consider to close this thread (9972828).