ana/err for Alpha Vms 7.3-2

[labadie_1]

I have an Alpha DS10 running Vms 7.3-2 which has crashed 3 times in the last days. As Ana/err is obsolete, I installed Webes, and tried various wsea commands, but I am still unable to do the equivalent of
$ ana/err/include=(bug,machine) sys$errorlog:errlog.old

and the same command with the 2 previous errlog.old

I get a lot of (uninteresting IMHO) stuff but nearly no dates and times.

Any idea ?

[Ian Miller.]

You could try
ANAL/ERROR/ELV TRANSLATE/INC=(BUG,MACH)

[labadie_1]

Ian

It gives me some info, but not at all as much as I had before for a bugcheck, with ana/err/full .

Rants:

1) who taked about "customer satisfaction" ? Compaq ?

2) non-regression tests, a new tool must provide at least as much info as the previous...

3) ergonomy, the Webes developers ever heard that word ?

Humm I feel better now :-)

[Jan van den Ende]

Gerard,

little consolation, but you are just one more voice in the choir :-(

I too would celebrate the guy(s) that re-instated ANA/ERR to its V6 behavior!

Proost.

Have one on me.

jpe

[John Abbott_2]

Have you tried DECevent ?

http://h18023.www1.hp.com/support/svctools/decevent/index.html#VMS

$dia/include=(bug,mach) should work. Works on V7.3-2, I think it'll give some reasonable errorlog translations for a DS10.

J.

[labadie_1]

John

dia/include=(bug,mach) sys$errorlog:errlog.old;-1

DECevent V3.4
SYSTEM >

Decevent 3.4 is not very useful...

I will dig in wsea and should eventually find the correct command

[Ian Miller.]

You could use ELV to translate the error log to the older format then use ANAL/ERROR.

See
HELP ANAL/ERROR/ELV TRANSLATE

[Arch_Muthiah]

Labadie,

Try this commands, but I never tried...

wsea bin input inputfile out outputfile include=â mchk_cpu, mchk_sysâ end=â some_dateâ

Create the new binary event log using the existing event log files using filtering keywords and required event types.

When you create the new event log, SEA checks all the events in the existing error log files against the filter statement, and only events that pass the filter get added to the new error log files.

you can use the following
--> filtering keywords
1. begin = any_date, end =any_date, since, today, yesterday
2. node = node_name
3. include = event_types

--> event types
1. mchk - All machine check events.
2. cam - SCSI entries logged by CAM logger
3. machine_checks (mchks)
4. panic - Crash restart, system panic, or user panic entries
5. etc.....

Archunan

[labadie_1]

Archunan

I did the following command, and I had the surprise to be asked if I wonted to overwrite my file a.txt (I understood this is a multi platform tool)

wsea bin input sys$errorlog:errlog.old;-1 out a.txt
/sys$sysroot/sysmgr/a.txt exists - overwrite([y]/n)? y
___.
RECOVERED FROM ERROR on July 19, 2006 8:21:01 AM CEST (5.171 sec elapsed)
Error encountetred during binary creation
Current Thread[main,5,main]
EXCEPTION java.io.IOException: Write error
at java.io.RandomAccessFile.writeBytes(byte[],int,int) (RandomAccessFile
.java)
at java.io.RandomAccessFile.write(byte[]) (RandomAccessFile.java:343) (p
c 5)
at com.compaq.svctools.ca.services.eventreaders.processors.BinaryCreator
.writeOneEvent (BinaryCreator.java:161) (pc 36)
at com.compaq.svctools.ca.services.eventreaders.processors.BinaryCreator
.writeEvents (BinaryCreator.java:135) (pc 58)
at com.compaq.svctools.ca.services.eventreaders.processors.BinaryCreator
.process (BinaryCreator.java:87) (pc 25)
at com.compaq.svctools.ca.cli.FilterCmd.execute (FilterCmd.java:88) (pc
110)
at com.compaq.svctools.ca.cli.CACLICommand.main(com.compaq.svctools.ca.c
li.CACLISyntax) (CACLICommand.java:96) (pc 6)
at com.compaq.svctools.ca.cli.FilterCmd.main(com.compaq.svctools.ca.cli.
CACLISyntax) (FilterCmd.java:50) (pc 28)
at com.compaq.svctools.ca.cli.CACLIInterpreter.main (CACLIInterpreter.ja
Exception encountered during binary creation processing: Write error

SYSTEM >

I will not say I am impressed by the quality of the product Webes / Wsea

[labadie_1]

Ian

I guess you mean convert and not translate ?

[Volker Halle · Accepted solution]

Gerard,

as you probably know, there are better tools for crash analysis than DECevent, ANAL/ERR or WEBES/SEA ;-)

What does TYPE CLUE$HISTORY say about your crashes ? And the CLUE$COLLECT:CLUE$node_ddmmyy_hhmm.LIS files ?

If it's not a MACHINECHK crash, just have a brief look at SDA> CLUE ERRLOG and if there was no HW error logged immediately preceeding the crash, forget about looking at ERRLOG.SYS

$ DIAGNOSE/INCL=(bug,mach) definitely should work. In my experience, it is still the best tool for having a first look at ERRLOG.SYS. It does work for my F8.3 MACHINECHK crash on an AlphaServer 1000. Did you look at the same ERRLOG.OLD file, for which ANAL/ERR/ELV did at least show the headers of the errors ? Does DIAGNOSE/SUMM/NOFULL ERRLOG.OLD report anything at all ?

Regarding WEBES/SEA, I would strongly suggest to use the WEB interface (http://your-node-ip-name:7902). Copy your ERRLOG.OLD file to svctools_home:[common.ca.userdata] first and select it in Add Logs to Tree and click on the filename in the left frame to start translation/analysis.

One of the (many) possible SEA syntaxes for translating machine check and bugcheck information would be:

$ wsea v /tra/inc=(panic,machine_checks) errlog.old

Volker.

[John Abbott_2]

As Volker says, dia/inc=(bug,mach) should work. I've only found it a bit problamatic when you start mixing other qualifers, i.e. the since or before qualifers.

Have you tried a crude $ DIA/OUT=T.TXT an done a search/edit of the output file. I know it's not ideal, but I'm hoping you'll get something to work with.

J.

[John Travell]

Have you ever had any valid crash dump files on this machine ? We will only fix crashes without a dumpfile if they are caused by hardware and you have machine checks in the errorlog.
Volker's comments about CLUE files do depend on there being a valid dump file. No valid dump, no CLUE info... However... if you only have the errorlog, this works for me on V8.2. I cannot check V7.3-2.
Between the lines is all one command line... (amend to point to the errorlog file you have)
-------------------------------------------
$ pipe ana/err/elv translate /include=bug sys$errorlog:errlog.old | search sys$pipe "Bugcheck" /window=(0,9)
-------------------------------------------

Can we go back to basics ? Do you have a valid dumpfile ?
The dumpfile will be either sys$sysroot:[sysexe]sysdump.dmp; or if bit 2 (DOSD) of (SYSGEN param) DUMPSTYLE is set, then in the exactly identical directory on whichever disk console env DUMP_DEV points to.
If neither of these files exist, and both sys$sysroot:[sysexe]pagefile.sys does, and SYSGEN SAVEDUMP is 1, you could find the dump there.
If you still have no dump file, then please create one.

If you DO have a valid dumpfile, can you attach both CLUE$COLLECT:CLUE$HISTORY.DAT;1 and CLUE$COLLECT:CLUE$node_ddmmyy_hhmm.LIS for at least one of the crashes.

[labadie_1]

Volker

some answers

>>>What does TYPE CLUE$HISTORY say about your crashes ?
%TYPE-W-SEARCHFAIL, error searching for SYS$COMMON:[SYSERR]CLUE$HISTORY.DAT;
-RMS-E-FNF, file not found


>>>CLUE$COLLECT:CLUE$node_ddmmyy_hhmm.LIS files ?

no files


>>>If it's not a MACHINECHK crash, just have a brief look at SDA> CLUE ERRLOG

this gives
%CLUE-W-NOTSUPP, not supported on running system

as I do not have a dump


>>>$ DIAGNOSE/INCL=(bug,mach) definitely should work.
DIAGNOSE/INCL=(bug,mach)

DECevent V3.4
SYSTEM >
There is a problem here


Did you look at the same ERRLOG.OLD file, for which ANAL/ERR/ELV did at least show the headers of the errors ?
I looked at all files errlog.sys, errlog.old, errlog.old;-1, errlog.old;-2

>>>Does DIAGNOSE/SUMM/NOFULL ERRLOG.OLD report anything at all ?
DIAGNOSE/SUMM/NOFULL ERRLOG.OLD

DECevent V3.4
_DIAGNOSE-FAT: An error occurred while executing a command ruleset
_DIAGNOSE-FAT: Translate - No files found or unknown command ' ERRLOG.OLD '

It seems I have a problem with Diagnose here

>>>Regarding WEBES/SEA, I would strongly suggest to use the WEB interface (http://your-node-ip-name:7902). Copy your ERRLOG.OLD file to svctools_home:[common.ca.userdata] first

Done, it offers me to select a username and then just hangs :-(


One of the (many) possible SEA syntaxes for translating machine check and bugcheck information would be:

>>>$ wsea v /tra/inc=(panic,machine_checks) errlog.old
This gives some info

wsea v /tra/inc=(panic,machine_checks) sys$errorlog:errlog.old;-2

Event: 28
Description: VMS Crash Restart Event at Wed 12 Jul 2006 07:24:51 GMT+02:00 from MHS083
File: /sys$errorlog/errlog.old;-2@mhs083.ikea.com
================================================================================

COMMON EVENT HEADER (CEH) V2.0
Event_Leader xFFFF FFFE
Header_Length 284
Event_Length 800
Header_Rev_Major 2
Header_Rev_Minor 1
OS_Type 2 -- OpenVMS
Hardware_Arch 4 -- Alpha
CEH_Vendor_ID 3,564 -- Hewlett-Packard Company
Hdwr_Sys_Type 34 -- Tsunami/Typhoon Corelogic
Logging_CPU 0 -- CPU Logging this Event
CPUs_In_Active_Set 1
Major_Class 37
Minor_Class 0
Entry_Type 37 -- VMS Crash Restart Event
DSR_Msg_Num 1,970 -- AlphaServer DS10
Chip_Type 11 -- EV67 - 21264A
CEH_Device 0
CEH_Device_ID_0 x0000 0000
CEH_Device_ID_1 x0000 0000
CEH_Device_ID_2 x0000 0000
Unique_ID_Count 1,213
Unique_ID_Prefix 1
Exact_Length 504
Num_Strings 6

TLV Section of CEH
TLV_DSR_String AlphaServer DS10 617 MHz
TLV_Sys_Serial_Num AY12502620
TLV_Time_as_Local Wed 12 Jul 2006 07:24:51 GMT+02:00
TLV_OS_Version V7.3-2
TLV_Computer_Name MHS083
Entry_Type 37

OpenVMS Crash Restart Event Data
KSP x0000 0000 7FF8 76E8
ESP x0000 0000 7FF8 C000
SSP x0000 0000 7FF9 CD00
USP x0000 0000 7AC9 9B80
R0 x0000 0000 0000 0001
R1 x0000 0000 0002 00E8
R2 x0000 0000 7FF6 40C0
R3 x0000 0000 0000 00FE
R4 xFFFF FFFF 8146 7750
R5 xFFFF FFFF 8156 DCC0
R6 xFFFF FFFF 8157 6B00
R7 xFFFF FFFF 8157 09C0
R8 xFFFF FFFF 8157 6A40
R9 x0000 0000 0000 00F6
R10 x0000 0000 0000 00FE
R11 x0000 0000 7FF8 76F0
R12 x0000 0000 7FF9 DDE8
R13 x0000 0000 0001 0130
R14 x0000 0000 0000 0000
R15 xA0A0 A0A0 0000 0000
R16 x0000 0000 0000 0474
R17 x0000 0000 0002 00D0
R18 x0000 0000 0000 039F
R19 xFFFF FFFF 8100 8000
R20 x0000 0000 0000 0000
R21 xFFFF FFFF FFFF FFFF
R22 xFFFF FFFF FFFF FF7F
R23 x0000 0000 0002 00E8
R24 x0000 0000 0000 0000
R25 xFFFF FFFF 8100 A600
R26 x0000 0000 0002 00E8
R27 x0000 0000 0000 0001
R28 x0000 0000 0003 0704
FP x0000 0000 7FF8 7EF0
SP x0000 0000 7FF8 76E8
PC x0000 0000 0003 0AFC
PSL x2800 0000 0000 1F03
PTBR x0000 0000 0002 E1D0
PCBB x0000 0000 1C66 E080
PRBR xFFFF FFFF 8142 0000
VPTB xFFFF FEFC 0000 0000
SCBB x0000 0000 0000 2428
SISR x0000 0000 0000 0000
ASN x0000 0000 0000 00B3
ASTSR_ASTEN x0000 0000 0000 000F
FEN x0000 0000 0000 0000
IPL x0000 0000 0000 001F
MCES x0000 0000 0000 0000
Crash_Code x0000 0474
Process_ID x0033 0192
Process_Name_Length 8
Process_Name STUDER_1
Crash_Code_String OPERATOR, Operator requested system shutdown

To John
This node is configured as many others, and so should be able to take a dump, but I will check.

Thanks

Gerard

[labadie_1]

John

1) ana/err/elv translate /include=(bug,machin)/summ sys$errorlog:errlog.old;-2
Output file SYS$OUTPUT: created at 20-JUL-2006 15:58:52.92


Output for SYS$SYSROOT:[SYSERR]ERRLOG.OLD;392

ERROR_LOG_SUMMARY______________________________________________________

Total number of events: 2
Number of the first event: 28
Number of the last event: 29
Earliest event occurred: 12-JUL-2006 07:24:51.81
Latest event occurred: 15-JUL-2006 15:12:11.00
Number of events by event class:
BUGCHECKS 1
MACHINE_CHECKS 1

2)
ana/err/elv translate /include=(bug,machin)/summ sys$errorlog:errlog.old;-1
Output file SYS$OUTPUT: created at 20-JUL-2006 15:59:31.63


Output for SYS$SYSROOT:[SYSERR]ERRLOG.OLD;393

ERROR_LOG_SUMMARY______________________________________________________

Total number of events: 1
Number of the first event: 20
Number of the last event: 20
Earliest event occurred: 17-JUL-2006 16:22:08.00
Latest event occurred: 17-JUL-2006 16:22:08.00
Number of events by event class:
MACHINE_CHECKS 1

3)

ana/err/elv translate /include=(bug,machin)/summ sys$errorlog:errlog.old;
Output file SYS$OUTPUT: created at 20-JUL-2006 16:00:03.74


Output for SYS$SYSROOT:[SYSERR]ERRLOG.OLD;394

ERROR_LOG_SUMMARY______________________________________________________

Total number of events: 1
Number of the first event: 19
Number of the last event: 19
Earliest event occurred: 18-JUL-2006 06:47:07.00
Latest event occurred: 18-JUL-2006 06:47:07.00
Number of events by event class:
MACHINE_CHECKS 1

[Volker Halle]

Gerard,

you can check SYS$MANAGER:CLUE$STARTUP_MHS083.LOG;* for any errors regarding processing of the dump file during startup. If you do not have a SYS$SYSTEM:SYSDUMP.DMP file, this won't help...

If you use DECevent and do NOT look at the current (default) SYS$ERRORLOG:ERRLOG.SYS file, you - apparently - need to specify the FULL filename path, i.e.

$ DIAGNOSE/SUMM/NOFULL SYS$ERRORLOG:ERRLOG.OLD

WEBES/SEA and ANAL/ERR/ELV seem to show one OPERATOR shutdown (12-JUL-2006 07:24) and 3 Machine Checks between 12-JUL 7:24 and 18-JUL 6:47

You need to at least be able to find and diagnose those 3 machine checks using SEA.
As wsea v /tra/inc=(panic,machine_checks) sys$errorlog:errlog.old;-2 has at least provided the bugcheck output, repeat this command without using the '/inc=(panic,machine_checks)' and capture the output using '/out=x.x'

If you have a valid SYS$SYSTEM:SYSDUMP.DMP file, it may still be possible that the system cannot take a valid dump due to some memory/HW problem. In this case, you need to capture the full console output.

Please check what AUTO_ACTION is set to. If it's NOT set to RESTART, you may also loose a dump, if the system just HALTs.

Volker.

[labadie_1]

Volker

I am not at the site at the moment, but I know that auto_action is set to HALT.

This may explain it.

[Volker Halle]

Gerard,


I have an Alpha DS10 running Vms 7.3-2 which has crashed 3 times in the last days


did the DS20 CRASH (and automatically reboot) or did it just HALT and some operator had to type >>> B or press the Restart button ? A small but significant difference !

A MCHECKPAL halt comes to mind. This one would cause the machine to halt (if AUTO_ACTION = HALT), but unfortunatley won't even log machine check errors (with AUTO_ACTION = RESTART), so your most valuable information would be the 3 machine check errors in your ERRLOG.OLD;* files right now.

Volker.

[labadie_1]

at least one time the ds10 hang ( I was logged, my session was unresponsive and it was impossible to telnet or set host) and was back 25 minutes later. As a normal reboot needs 3 or 4 minutes, and auto_action is halt, somebody rebooted. I will ask the people at the remote node.