- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: CRASH ANALYSIS of VAX/VMS server
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 12:25 AM
02-01-2006 12:25 AM
CRASH ANALYSIS of VAX/VMS server
Please help...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 01:08 AM
02-01-2006 01:08 AM
Re: CRASH ANALYSIS of VAX/VMS server
$ show system/noproc
$ diag/sinc=28-jan-2006/incl=(cpu,mem,bug,mach)
$ anal/cras sys$system:sysdump.dmp
SDA> clue crash
SDA> ^Z
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 01:39 AM
02-01-2006 01:39 AM
Re: CRASH ANALYSIS of VAX/VMS server
this is a VAX, so there is no SDA> CLUE...
Let's start with:
$ ANAL/CRASH SYS$SYSTEM:SYSDUMP.DMP
SDA> READ/EXEC
SDA> SHOW CRASH
SDA> SHOW STACK
$ ANAL/ERR/SINCE=
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 01:46 AM
02-01-2006 01:46 AM
Re: CRASH ANALYSIS of VAX/VMS server
On VAX/VMS CLUE is a seperate program so instead of using the CLUE command after ANAL/CRASH you have to
MCR CLUE SYS$SYSTEM:SYSDUMP.DMP
you may already have a file in
SYS$MANAGER:CLUE*.LIS
(check the dates).
Purely Personal Opinion
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 02:26 AM
02-01-2006 02:26 AM
Re: CRASH ANALYSIS of VAX/VMS server
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 02:27 AM
02-01-2006 02:27 AM
Re: CRASH ANALYSIS of VAX/VMS server
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 02:28 AM
02-01-2006 02:28 AM
Re: CRASH ANALYSIS of VAX/VMS server
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 02:28 AM
02-01-2006 02:28 AM
Re: CRASH ANALYSIS of VAX/VMS server
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 04:07 AM
02-01-2006 04:07 AM
Re: CRASH ANALYSIS of VAX/VMS server
it's the same old problem, isn't it ?!
See threads:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=938939
and
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=929654
An unexpected HALT on a VAX 4000-105 type of machine running V7.1, apparently running the same kind of application [AGMS.MCSYS...]*.EXE. Are these really different systems ? Or is it the same physical system booted with different node names from time to time ?
The crash (and therefore the CLUE file) does NOT contain the actual HALT PC and PSL. To capture these values, you need to record the console output from this system. Only the correct HALT-PC would allow a check, whether there really was a HALT instruction (or a binary NULL) in the instruction stream being executed.
What's unusual are the 3 non-fatal HALT bugchecks preceeding the fatal one (within a 4 minute timeframe).
The ICCS error bit is set.
CESR shows a CP1 Interrupt Vector Read Timeout
It may be hardware. See comments in the previous threads.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 04:22 AM
02-01-2006 04:22 AM
Re: CRASH ANALYSIS of VAX/VMS server
There 12 such servers(3 for each area) having same processes and same functionality with more or less same configuration. The only difference is no. of data points configured.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 05:08 AM
02-01-2006 05:08 AM
Re: CRASH ANALYSIS of VAX/VMS server
$ ANAL/CRASH SYS$SYSTEM:
SDA> READ SYS$SYSTEM:SYSDEF
SDA> FORM @EXE$GL_RPB/TYPE=RPB
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 08:05 PM
02-01-2006 08:05 PM
Re: CRASH ANALYSIS of VAX/VMS server
Please find the RPB report
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2006 10:35 PM
02-01-2006 10:35 PM
Re: CRASH ANALYSIS of VAX/VMS server
A short test with a program-incuded HALT restart crash under OpenVMS VAX V7.3 on a CHARON-VAX 4000-108A shows that the only reliable info about the real HALT PC is on the console terminal !
CHARON $ run halt
?06 HLT INST
PC = 00000218
Restarting system software.
**** Fatal BUG CHECK, version = V7.3 HALT, Halt instruction restart
...
The data in the crash and in the CLUE file produced afterwards do not provide the actual HALT PC. They look similar to your case, but at least the HALT code in AP (and on the INT stack) is correctly stored as 00000006 (HALT instruction executed in kernel mode).
SDA> SHOW CALL
SDA> SHOW CALL/NEXT
SDA> SHOW CALL/NEXT
at least allowed to traverse the call stack. IF the problem is indeed a software-induced HALT, this may also work in your case. Please provide the SHOW CALL data (repeat SHOW CALL/NEXT until the Return PC is in P0 space).
Consider to connect some hardcopy terminal or console manager type application to the console terminal to capture the real HALT pc and reason code.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2006 08:10 PM
02-02-2006 08:10 PM
Re: CRASH ANALYSIS of VAX/VMS server
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2006 08:51 PM
02-02-2006 08:51 PM
Re: CRASH ANALYSIS of VAX/VMS server
There should be another 2 such application error messages from processes DMEDIS and PPTTBF within the last 5 minutes before the crash (they could be seen in the ANAL/ERR output). Can you find them ?
This traceback data indicates, that the the instruction at PC=867C9EC0 tried to access some P1 space address (7FF89D70 - most likely a stack address, this address is also in R9) and failed. The failing PC is in MESSAGE_ROUTINES+22C0
Could this be some kind of stack overflow or underflow ?
The crash-PC reported in the crash (on the kernel stack) is 195FEB, quite near to 195D37 on top of the traceback call stack.
Can you do the following in SDA (in the dump):
SDA> READ/EXEC
SDA> EXA/INS 867C9EC0
SDA> EXA/INS 195D37
SDA> EXA/INS 195FEB
SDA> SHOW CALL
SDA> SHOW CALL/NEXT
SDA> SHOW CALL/NEXT
SDA> SHOW CALL/NEXT
You may also want to run your application processes with process dump enabled. Put a SET PROC/DUMP in their login procedures or run them with RUN/DUMP. This would cause process dump files to be written on unhandled exceptions.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2006 09:22 PM
02-02-2006 09:22 PM
Re: CRASH ANALYSIS of VAX/VMS server
Please find the SDA output asked by you
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2006 09:48 PM
02-02-2006 09:48 PM
Re: CRASH ANALYSIS of VAX/VMS server
EXE$PUTMSG+23: CMPZV #10,#0C,(R9),#01
with R9 pointing to an in-accessible address in P1 space (stack ?), which should contain the message code to be reported.
Your application seems to have called SYS$PUTMSG (from PC=005E8ED3). Saved R3 = 0001827A = %RMS-E-EOF - this could make sense.
The ACCVIO in EXE$PUTMSG seems to have triggered a call to an application condition handler (?).
00195D37: BLBS R0,00195D4D looks like a valid typical return address. Where does the call go ?
Please try:
SDA> EXA/INS 00195D37-n;20
Start with n=10 and vary n, until you get a valid instruction stream.
SDA> exa/ins 195feb
00195FEB: XFC - looks like the entry mask, please also show:
SDA> EXA/INS 195FEB;50
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2006 10:34 PM
02-02-2006 10:34 PM
Re: CRASH ANALYSIS of VAX/VMS server
SDA> EXA CTL$AL_STACK;10 ! bottom of stacks
SDA> EXA CTL$AL_STACKLIM;10 ! top of stacks
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 12:57 AM
02-03-2006 12:57 AM
Re: CRASH ANALYSIS of VAX/VMS server
Pls find the report attached as aked by you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 01:47 AM
02-03-2006 01:47 AM
Re: CRASH ANALYSIS of VAX/VMS server
SDA> exa/ins 00195d37-10;20
00195D27: BRW 00195DEF
00195D2A: PUSHL #10000002
00195D30: CALLS #01,@0019C250
00195D37: BLBS R0,00195D4D < Return PC
Where does that call go ?
SDA> EXA 19C250
SDA> EXA @19C250
SDA> EXA/INS @19C250;10
The following is the code stream from the interrupted PC found on the kernel stack:
SDA> exa/ins 195feb;50
%SDA-W-INSKIPPED, unreasonable instruction stream - 1 bytes skipped
00195FEC: HALT
00195FED: PUSHL R0
00195FEF: PUSHL R1
00195FF1: ADDL3 #10,08(AP),R1
00195FF6: JSB @#EXE$ALONONPAGED
00195FFC: BLBC R0,00196032
Please also provide the stack limits (see my previous entry).
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 02:24 AM
02-03-2006 02:24 AM
Re: CRASH ANALYSIS of VAX/VMS server
SDA> sho call/next
Call Frame Information
----------------------
Call Frame Generated by CALLS Instruction
Condition Handler 7FF751D4 00000000
SP Align Bits = 00 7FF751D8 2FFC0000
Saved AP 7FF751DC 7FF75248
Saved FP 7FF751E0 7FF75224
Return PC 7FF751E4 005E8ED3 <
R2 7FF751E8 005D8E4C
R3 7FF751EC 0001827A
R4 7FF751F0 0000000F
R5 7FF751F4 7FF75298
R6 7FF751F8 005D8EE4
R7 7FF751FC 005D8E08
R8 7FF75200 005E8000
R9 7FF75204 00602484
R10 7FF75208 7FF75627
R11 7FF7520C 7FF7562D
Align Stack by 0 Bytes =>
Argument List 7FF75210 00000001
7FF75214 7FF75218
The MSGVEC argument starts at 7FF75218. Please examine:
SDA> EXA 7FF75218;50
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 02:29 AM
02-03-2006 02:29 AM
Re: CRASH ANALYSIS of VAX/VMS server
Pls find the stack limit
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 02:34 AM
02-03-2006 02:34 AM
Re: CRASH ANALYSIS of VAX/VMS server
SDA > exa 7FF75218;50
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 04:26 AM
02-03-2006 04:26 AM
Re: CRASH ANALYSIS of VAX/VMS server
But the MSGVEC argument to the SYS$PUTMSG call looks VERY suspicious:
The call frame to SYS$PUTMSG from your code (return address = 005E8ED3) presents ONE argument to SYS$PUTMSG - the argument list is on the stack:
Argument List: 7FF75210 00000001
______________ 7FF75214 7FF75218
The first argument to SYS$PUTMSG is the message vector MSGVEC passed by reference. So the MSGVEC starts at 7FF75218 and looks like this:
7FF75218: 7FF7562D <<< def msg options & arg count !
7FF7521C: 0001827A <<< message code = %RMS-E-EOF
7FF75220: 00000000 <<< STV value
7FF75224: 00000000
...
The argument count field, the low 16 bits of the first longword in the message vector is DEFINITELY WRONG ! It should have been 0001 ! EXE$PUTMSG 'walks' the argument vector based on the argument count. It may well walk into in-accessible address space based on that incorrect value of 562D.
Please use SDA> SHOW PROC/IMAGE to find out, which code is at address 005E8ED3, find the map and listings and the programmer (!) and check the construction of the MSGVEC for this call to SYS$PUTMSG.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2006 04:39 AM
02-03-2006 04:39 AM
Re: CRASH ANALYSIS of VAX/VMS server
SDA> EXA 19C250
SDA> EXA @19C250
SDA> EXA/INS @19C250;10
This could finally even lead us to determine the reason for the HALT
Volker.