crash ovms 8.3 integrity

[Edgar Ulloa]

Hi

yesterday was a abrupt crash in only one node of my cluster, basicly was a 34078720 PFNs, discontiguous memory.

The last sentences

Failing Instruction:
TCPIP$TNDRIVER+198F1: ld4 r18 = [r9]

Instruction Stream (last 20 instructions):
TCPIP$TNDRIVER+198A0: ld4 r17 = [r16]
TCPIP$TNDRIVER+198A1: add r16 = 03FC, r5 ;;
TCPIP$TNDRIVER+198A2: sxt4 r17 = r17
TCPIP$TNDRIVER+198B0: add r8 = 0001, r0
TCPIP$TNDRIVER+198B1: sxt4 r9 = r9
TCPIP$TNDRIVER+198B2: nop.b 000000 ;;
TCPIP$TNDRIVER+198C0: st1 [r17] = r3
TCPIP$TNDRIVER+198C1: add r17 = 03FC, r5
TCPIP$TNDRIVER+198C2: nop.i 000000 ;;
TCPIP$TNDRIVER+198D0: ld4 r24 = [r23]
TCPIP$TNDRIVER+198D1: add r23 = 03EC, r5 ;;
TCPIP$TNDRIVER+198D2: sxt4 r24 = r24 ;;
TCPIP$TNDRIVER+198E0: add r24 = 0001, r24 ;;
TCPIP$TNDRIVER+198E1: st4 [r22] = r24
TCPIP$TNDRIVER+198E2: add r9 = 0058, r9

I have installed
HP I64VMS VMS83I_DRIVER V1.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_FIBRE_SCSI V6.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_LAN V7.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_LIBRAR V2.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_MOUNT96 V4.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_RMS V3.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_SYS V5.0 Patch Install Val 11-DEC-2007
HP I64VMS VMS83I_UPDATE V5.0
HP I64VMS TCPIP V5.5-11ECO1 Full LP

some one knows any idea if tcpip needs a patch?

regards

[Ian Miller.]

Can you post the header of the clue file with the bugcheck etc

[Edgar Ulloa]

System crash information
------------------------

Time of system crash: 19-AUG-2009 15:32:16.67
Version of system: OpenVMS I64 Operating System, Version V8.3

System Version Major ID/Minor ID: 3/0
VMScluster node: YIRE2
System type: HP rx2620 (1.60GHz/6.0MB)

CPUs are not thread-capable

Primary CPU ID: 000 (0.)
Crash CPU ID: 001 (1.)

Bitmask of active CPUs: 00000000.00000003
Bitmask of available CPUs: 00000000.00000003
CPU bugcheck codes:
CPU 001 -- database address 8821F280 -- INVEXCEPTN, Exception while abov
e ASTDEL

1 other -- CPUEXIT, Shutdown requested by another CPU
CPU 000 -- database address 880DA000System State at Time of Original Exception
------------------------------------------

Exception Frame at FFFFFFFF.9223D970
------------------------------------

IPL = 08
TRAP_TYPE = 00000008 Access control violation fault
IVT_OFFSET = 00001400 Data Nested TLB Fault
IIP = FFFFFFFF.816607F0 TCPIP$TNDRIVER+198F0
IIPA = FFFFFFFF.816607F0 TCPIP$TNDRIVER+198F0
IFA = 00000000.00000058

IPSR = 00001210.08026010 RT TB LP DB SI DI PP SP D
FH DFL DT PK I IC MFH MFL AC BE UP
1 0 0 0 0 0 0 0 0
0 1 0 1 1 0 1 0 0 0
IA BN ED RI SS DD DA ID I
T MC IS CPL
0 1 0 1 0 0 0 0 1
0 0 0
System State at Time of Original Exception
------------------------------------------

PREVSTACK = 00
BSP = FFFFFFFF.92228C10
BSPSTORE = FFFFFFFF.92228B90
BSPBASE = FFFFFFFF.92228B90
RNAT = 00000000.00000000

RSC = 00000000.00000003 LOADRS BE PL MODE
0000 0 0 Eager

PFS = 00000000.00000797 PPL PEC RRB.PR RRB.FR R
RB.GR SOR SOL SOF
0 0. 0. 0.
0. 0. 15. (32-46) 23. (32-54)

FLAGS = 00
STKALIGN = 000002D0
IHA = FFFFFFFF.7FF38000
INTERRUPT_DEPTH = 02

Press RETURN for more.
System State at Time of Original Exception
------------------------------------------
PREDS = 00000000.0001D527 P0 P8 P16 P24
P32 P40 P48 P56
11100100 10101011 10000000 000
00000 00000000 00000000 00000000 00000000

ISR = 00000A04.00000000 ED EI SO NI IR RS SP NA R
W X CODE
1 1 0 0 0 0 0 0 1
0 0 0000

ITIR = 00000000.00000034 KEY PS
000000 0D

IFS = 80000000.00000818 Valid RRB.PR RRB.FR R
RB.GR SOR SOL SOF
1 0. 0.
0. 0. 16. (32-47) 24. (32-55)

B0 = FFFFFFFF.81663130 TCPIP$TNDRIVER+1C230
System State at Time of Original Exception
------------------------------------------
B1 = 00000000.00000000
B2 = 00000000.00000000
B3 = 00000000.00000000
B4 = 00000000.00000000
B5 = 00000000.00000000
B6 = FFFFFFFF.81660570 TCPIP$TNDRIVER+19670
B7 = FFFFF802.88800880 SYS$PAL_REMQUEL_D_C

GP = FFFFFFFF.90097E00 TCPIP$TNDRIVER_GP
R2 = 00000000.00000001
R3 = 00000000.00000001
R4 = FFFFFFFF.88B3CE40
R5 = FFFFFFFF.88B3C940
R6 = FFFFFFFF.88AF1347
R7 = FFFFFFFF.8FE984E0 TCPIP$TNDRIVER+806E0
R8 = 00000000.00000001
R9 = 00000000.00000058
R10 = 00000000.00000043
R11 = FFFFFFFF.8F8178F8 SMP$GQ_DEBUG

System State at Time of Original Exception
------------------------------------------

KSP = FFFFFFFF.9223DC40

R13 = 00000000.00000000
R14 = 00000000.00000000
R15 = FFFFFFFF.8FE97514 TCPIP$TNDRIVER+51B14
R16 = FFFFFFFF.88B3CD3C
R17 = FFFFFFFF.88B3CD3C
R18 = FFFFFFFF.88B3CD34
R19 = FFFFFFFF.88B3CD3C
R20 = 00000000.00000000
R21 = 00000000.00000000
R22 = FFFFFFFF.88B3CD2C
R23 = FFFFFFFF.88B3CD2C
R24 = FFFFFFFF.88E46A93
R25 = 00000000.00000002
R26 = 00000000.00000043
R27 = 00000000.00000001
R28 = 00000000.000001AB
System State at Time of Original Exception
------------------------------------------
R29 = FFFFFFFF.9223DC60
R30 = FFFFFFFF.8F9D9F78 SYSTEM_PRIMITIVES_MIN+00225578
R31 = FFFFFFFF.8F9E1350 SYSTEM_PRIMITIVES_MIN+00250B50

R32 = FFFFFFFF.88AF12F8
R33 = 00000000.00000043
R34 = 00000000.00000001
R35 = 00000000.00000043
R36 = FFFFFFFF.8F8178F8 SMP$GQ_DEBUG
R37 = FFFFFFFF.8F9D9F78 SYSTEM_PRIMITIVES_MIN+00225578
R38 = FFFFFFFF.8F9E1350 SYSTEM_PRIMITIVES_MIN+00250B50
R39 = FFFFFFFF.8166B950 TCPIP$TNDRIVER+24A50
R40 = FFFFFFFF.90097E00 TCPIP$TNDRIVER_GP
R41 = FFFFFFFF.9223DC60
R42 = FFFFFFFF.81663130 TCPIP$TNDRIVER+1C230
R43 = 00000000.00000797
R44 = FFFFFFFF.9223DC40
R45 = FFFFFFFF.8FE97510 TCPIP$TNDRIVER+51B10
R46 = FFFFFFFF.88B3C940
System State at Time of Original Exception
------------------------------------------
R47 = FFFFFFFF.88B3C940

R48/OUT0 = 00000000.000010A9
R49/OUT1 = FFFFFFFF.9223DC40
R50/OUT2 = FFFFFFFF.88C89700
R51/OUT3 = FFFFFFFF.8F9D9F78 SYSTEM_PRIMITIVES_MIN+00225578
R52/OUT4 = 00000000.00000008
R53/OUT5 = 00000000.00000008
R54/OUT6 = FFFFFFFF.80225670 SMP$ACQUIREL_C+00170
R55/OUT7 = 00000000.000011AB

NATMASK = 002E
NATS = 00000000.00000000
CSD = 00000000.00062974
SSD = 00000000.7AD23430
LC = 00000000.00000000
EC = 00000000.00000000

System State at Time of Original Exception
------------------------------------------
FPSR = 0009804C.8A70033F SF3 SF2 SF1 SF0 TRAPS
004C 004C 114E 000C 3F

F6 = 1003E.00000000.00000000
F7 = 1003E.00000000.00001000
F8 = 1003E.00000000.00000000
F9 = 0FFF6.80000000.00000000
F10 = 1003E.00000000.0A011000
F11 = 0FFDD.80000000.00000000

PPREVMODE = 00

System State at Time of Original Exception
------------------------------------------
Instruction Stream:
-------------------
{ .mii
TCPIP$TNDRIVER+198D0: ld4 r24 = [r23]
add r23 = 03EC, r5 ;;
sxt4 r24 = r24 ;;
}
{ .mmi
TCPIP$TNDRIVER+198E0: add r24 = 0001, r24 ;;
st4 [r22] = r24
add r9 = 0058, r9
}
{ .mmi
TCPIP$TNDRIVER+198F0: add r22 = 03EC, r5 ;;
PC => ld4 r18 = [r9]
nop.i 000000 ;;
}
{ .mii
TCPIP$TNDRIVER+19900: nop.m 000000
System State at Time of Original Exception
------------------------------------------
sxt4 r18 = r18 ;;
add r18 = 0001, r18 ;;
}
{ .mmi
TCPIP$TNDRIVER+19910: st4 [r9] = r18, 1A8 ;;
ld4 r24 = [r23]
nop.i 000000 ;;
}

Signal Array at: FFFFFFFF.9223D920
----------------------------------
Length = 00000005
Type = 0000000C
Arg = 00000000.00000000
Arg = 00000000.00000058
Arg = FFFFFFFF.816607F1 TCPIP$TNDRIVER+198F1
Arg = 00000000.00000800
%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=000000000000
0058, PC=FFFFFFFF816607F1, PS=00000800
CPU 001 Processor state at time of INVEXCEPTN bugcheck
------------------------------------------------------


CPU 001 reason for Bugcheck: INVEXCEPTN, Exception while above ASTDEL


Process currently executing on this CPU: None


Current IPL: 8 (decimal)


CPU database address: 8821F280


CPUs Capabilities: QUORUM,RUN


Exception Frame Summary:
CPU 001 Processor state at time of INVEXCEPTN bugcheck
------------------------------------------------------
Exception Frame Type Stack IIP / Ret_Addr Trap_Typ
e / Service_Number
----------------- ---- ----- ----------------- --------
------------------
FFFFFFFF.9223D360 ORIGINAL_INTSTK System FFFFFFFF.803CBC10 00000041
Bugcheck Breakpoint Trap
FFFFFFFF.9223D970 INTSTK System FFFFFFFF.816607F0 00000008
Access control violation fault
FFFFFFFF.9223DD00 INTSTK System FFFFFFFF.80572D30 00000061
Interprocessor Interrupt

CPU 001 Processor state at time of INVEXCEPTN bugcheck
------------------------------------------------------

Exception Frame at FFFFFFFF.9223D360
------------------------------------

IPL = 08
TRAP_TYPE = 00000041 Bugcheck Breakpoint Trap
IVT_OFFSET = 00002C00 Break Instruction
IIP = FFFFFFFF.803CBC10 EXCEPTION+97E10
IIPA = FFFFFFFF.803CBC00 EXCEPTION+97E00
IFA = 00000000.00000058

IPSR = 00001010.08026010 RT TB LP DB SI DI PP SP D
FH DFL DT PK I IC MFH MFL AC BE UP
1 0 0 0 0 0 0 0 0
0 1 0 1 1 0 1 0 0 0
IA BN ED RI SS DD DA ID I
T MC IS CPL
0 1 0 0 0 0 0 0 1
0 0 0

CPU 001 Processor state at time of INVEXCEPTN bugcheck
------------------------------------------------------

PREVSTACK = 00
BSP = FFFFFFFF.92228D88
BSPSTORE = FFFFFFFF.92228BA8
BSPBASE = FFFFFFFF.92228BA8
RNAT = 00000000.00000000

RSC = 00000000.00000003 LOADRS BE PL MODE
0000 0 0 Eager

PFS = 00000000.00000E24 PPL PEC RRB.PR RRB.FR R
RB.GR SOR SOL SOF
0 0. 0. 0.
0. 0. 28. (32-59) 36. (32-67)

FLAGS = 00
STKALIGN = 000002D8
IHA = FFFFFFFF.7FF38000
INTERRUPT_DEPTH = 02
CPU 001 Processor state at time of INVEXCEPTN bugcheck
------------------------------------------------------
PREDS = 00000000.000154A7 P0 P8 P16 P24
P32 P40 P48 P56
11100101 00101010 10000000 000
00000 00000000 00000000 00000000 00000000

ISR = 00000000.00000000 ED EI SO NI IR RS SP NA R
W X CODE
0 0 0 0 0 0 0 0 0
0 0 0000

ITIR = 00000000.00000034 KEY PS
000000 0D

IFS = 80000000.00000E24 Valid RRB.PR RRB.FR R
RB.GR SOR SOL SOF
1 0. 0.
0. 0. 28. (32-59) 36. (32-67)

B0 = FFFFFFFF.803C0A60 EXCEPTION+8CC60
CPU 001 Processor state at time of INVEXCEPTN bugcheck
------------------------------------------------------
B1 = 00000000.00000000
B2 = 00000000.00000000
B3 = 00000000.00000000
B4 = 00000000.00000000
B5 = 00000000.00000000
B6 = FFFFFFFF.803C0040 EXCEPTION+8C240
B7 = FFFFF802.88800880 SYS$PAL_REMQUEL_D_C

GP = FFFFFFFF.8FC4A400 EXCEPTION_GP
R2 = 0009804C.8A70033F
R3 = FFFFFFFF.9223D8F0
R4 = FFFFFFFF.9223D6B0
R5 = FFFFFFFF.9223D908
R6 = FFFFFFFF.9223D970
R7 = FFFFFFFF.8FE984E0 TCPIP$TNDRIVER+806E0
R8 = 00000000.0000043C
R9 = 00000000.0000000C
R10 = 00000000.00000001
R11 = FFFFFFFF.9223DAD8
CPU 000 Processor state at time of CPUEXIT bugcheck
---------------------------------------------------


CPU 000 reason for Bugcheck: CPUEXIT, Shutdown requested by another CPU


Process currently executing on this CPU: PLANES


Current image file: $1$DGA102:[PRODUCTO.COM]VAXLINK2.EXE;4


Current IPL: 31 (decimal)


CPU database address: 880DA000



REGARDS

[Hoff · Accepted solution]

Regarding whether your box is current on patches and rather than asking (I really wish OpenVMS could answer this question for you; the current manual patch process is archaic), here is the Patch FAQ, and here is how to install kits using VMSINSTAL and PCSI, respectively.

http://labs.hoffmanlabs.com/node/348
http://labs.hoffmanlabs.com/node/570

Here's the direct ftp link to the patch area:

ftp://ftp.itrc.hp.com/openvms_patches/

Depending on what you are up to, you'll look in the OS area or in the layered products area. Here, it's the LP area.

And no, having just looked, you're not current.

HP-I64VMS-TCPIP-V0505-11ECO3-1.ZIPEXE

or better, V5.6 and:

HP-I64VMS-TCPIP-V0506-9ECO4-1

[Hoff]

ps: The VAXLINK2 stuff is (was?) usually from the WRQ Reflections package; check with the owners (Attachmate?) of that stuff.

I'm guessing this package might have been image-translated, too.

Alternatively, there are other mechanisms for transferring files around. Packages such as the freeware FileZilla tool can potentially be used here, given you're probably using Microsoft Windows clients here. There are others.

[Edgar Ulloa]

thanks allot

This process belongs at my aplication, some user was forgeting close the aplication las night and hung up, the memory was over and crashing the machine.

Regards

[Edgar Ulloa]

thanks allot Hoff and Ian

This process belongs at my aplication, some user was forgeting close the aplication las night and hung up, the memory was over and crashing the machine.

Regards

[Hein van den Heuvel]

It's good that you can relate the crash to an event, but hopefully you are not going to stop at that point right?
The system should _not_ crash just because a process did not exit 'nicely'. That is NOT a valid/acceptable reason.
If this VAXLINK process is running privileged code, then it's support should be engaged ASAP. If it is just a TCP/IP user, then HP support should be requested for thsi problem.

Probably stating the obvious here...

Cheers,
Hein