Troubleshooting DELPEN process

[Volker Halle]

Richard,

and what is at SDA> EXA/INS 0030E060 - could this be in your image in P0 space ?

Volker.

[Richard Brodie_1]

It looks like it's in a protected shareable image, from a third party supplier.

[Volker Halle]

Richard,

now you know whom you could try to ask next...

Let's make this troubleshooting example a bit more detailled. PWAIT$SDA could be amended to provide the adresses and symbolization of the exec (and kernel) mode rundown handlers for the given process, if ERDACT is set (as indicated by the PWAIT$SDA message: Process exec mode rundown is active).

During exec mode rundown, the SYS$ERNDWN system service (in module [SYS]SYSRUNDWN) invokes the exec mode rundown services in exec mode after setting PCB$V_ERDACT. Note that ERDACT will only be CLEARED at the end of kernel mode rundown, so a problem in either exec mode rundown or kernel mode rundown may exist here !

The APLD (Activated Privileged Library Dispatch vector) can be formatted in SDA like this:

$ CREATE APLDDEF.MAR

; assemble with $ mac/migr aplddef+sys$library:lib/libr
; link with     $ link/sym aplddef/noexe
        $aplddef GLOBAL
        .end

<CTRL-Z>
$ mac/migr aplddef+sys$library:lib/libr
$ link/noexe/sym aplddef
%ILINK-W-USRTFR, image NL:[].EXE; has no user transfer address

$ ANALYZE/SYS

SDA> SET PROC/ind=<pid-of-hung-process>

SDA> READ APLDDEF

SDA> FORMAT @ctl$a_dispvec/type=apld           ! format the APLD

...

00000000.7FFB7CB0   APLD$PS_EXEC_RUNDOWN_VECTOR               0030E060      <<< from your case... 
00000000.7FFB7CB4                                   00000000
00000000.7FFB7CB8                                   00000000.00000000
       ...                                                 ...
00000000.7FFB7D58   APLD$PS_KERN_RUNDOWN_VECTOR              8EFA11D0      SYS$IPC_SERVICES+67380

00000000.7FFB7D5C                                   0030E060                                                              <<< from your case...
00000000.7FFB7D60                                   00000000.00000000
       ...                                                 ...

This indicates, that there is an exec AND kernel mode rundown handler within the P0 address space of your image. Use SDA> SHOW PROC/IMAGE to find out, in which image or library that address resides.

If you can find out, in which mode the processes is hanging, this could provide further evidence as to which rundown handler may be executing. Try

SDA> SHOW EXCEPTION

Volker.

[Richard Brodie_1]

Morning, Volker, and thanks for your help so far,

So, it has the same routine as both an exec and a kernel mode rundown handler, plus SYS$IPC_SERVICES+67380 (which I presume is ICC related). 

SDA> show exc

Exception Frame Summary
-----------------------

 Exception Frame    Type              Stack          IIP / Ret_Addr     Trap_Type / Service_Number
-----------------   ----              -----         -----------------   --------------------------
00000000.7FF43B30   SSENTRY           Kernel        FFFFFFFF.806016B0   0100015D  SYS$SYNCH_INT
00000000.7FF43D50   SSENTRY           Kernel        FFFFFFFF.80B6A370   01000028  SYS$DELPRC
00000000.7FF43F40   SSENTRY           Kernel        FFFFFFFF.80B68BE0   0100018E  SYS$EXIT_INT
%SDA-W-NOREAD, unable to access location 00000000.7FF43FE0
00000000.7FF67F40   SSENTRY           Executive     FFFFFFFF.805D4110   01000197  SYS$HIBER_INT

[Volker Halle]

Richard,

try SDA> CLUE REGISTER  ! trying to determine the current execution context...

Try SDA> SHOW EXCEPTION 7FF43B30 ! to display the register values

The exception frames can be located o.k. on the current kernel stack.

If you somehow have to reboot the system, please force a crash to document this process state for later analysis.

Volker.

[Richard J Maher]

Is it not time for a support call to your third-party UWSS vendor? Or look at the source code if you have it?

Maybe their rundown handler is doing a send/receive/transceive in a rundown handler that can be completed somehow avoiding a reboot.

[Richard Brodie_1]

Richard,

I could try that: they do provide good support but they aren't really kernel gurus.

Also, the ICC calls are from my code, so if the problem is in there, it's not their responsibility anyway. I presume the system hooks its own rundown handler for that: it's not my doing.

[Richard Brodie_1]

Unfortunately, it doesn't seem to want to give up registers on the live system:

SDA> set proc/in=177
SDA> clue register
SDA> clue register

System Service Entry Frame at 00000000.7FF43B30
-----------------------------------------------

    IPL              =                 00
    SERVICE_NUMBER   =           0100015D      SYS$SYNCH_INT
    RET_ADDR         =  FFFFFFFF.806016B0      EXE_STD$SYNCH_LOOP_C+001A0

    PREVSTACK        =                 00
    BSP              =  00000000.7FF2E7C8
    BSPSTORE         =  00000000.7FF2E6E8
    BSPBASE          =  00000000.00000000
    RNAT             =  00000000.00000000

    RSC              =  00000000.00000000      LOADRS   BE   PL   MODE
                                               0000     0    0    Enforced lazy

    PFS              =  00000000.00000E24      PPL    PEC    RRB.PR   RRB.FR   RRB.GR     SOR       SOL           SOF
                                               0       0.       0.       0.       0.       0.    28. (32-59)   36. (32-67)

    FLAGS            =                 00
    STKALIGN         =           00000160

    PPREVMODE        =                 00

[Volker Halle]

Richard,

the $ICC Kernel-Mode Rundown Routine (ICC_KMODE_RUNDWN in module [IPC]ICC_SUBS) will try to disconnect all connections and close all associations and clean up the ICC data structures for the current process/image. If successful, it will clear the process entry in the ICCPDB_VECTOR.

SDA> EXAM @ICC$GL_ICC_PDB_VECTOR;4*(@SGN$GW_MAXPRCCT&ffff)

There should one 1 entry (longword) for each process on the system, indexed by PID. If you find a non-zero entry for the current process, it will indicate, that ICC has not (yet) been completely run down for this process.

Volker.

[Richard Brodie_1]

Volker,

Yes, there is still an entry in the ICCPDB_VECTOR.