Operating System - OpenVMS
1748161 Members
3799 Online
108758 Solutions
New Discussion

Re: Troubleshooting DELPEN process

 
Richard Brodie_1
Honored Contributor

Troubleshooting DELPEN process

I've got an unusual DELETE pending process; it's delete pending but stuck in LEF state, with no channels busy (OpenVMS V8.3/I64). Any thoughts on how to troubleshoot it? It could possibly be related to ICC communication, since it uses those calls.

 

 

21 REPLIES 21
Volker Halle
Honored Contributor

Re: Troubleshooting DELPEN process

Richard,

 

there is PWAIT$SDA, a SDA extension provided by Ian Miller, which helps to diagnose process wait states.

 

http://www.encompasserve.org/~miller/ has the most recent version, but there are also versions on the OpenVMS Freeware CDs.

 

I've also done a DECUS presentation some years ago about Analyzing Process Hangs, which includes a DELPEN example (sorry, it's in German, but you will get the idea ;-)

 

http://www.decus.de/slides/sy2007/19_04/3g03.pdf

 

Volker.

John Gillings
Honored Contributor

Re: Troubleshooting DELPEN process

Richard,

 

   I'm intrested in the possible ICC connection. Do you know what's happening with the ICC association? Could it be clogged up with messages? Is this process a reader, writer, or both? If you trace the stack from SDA are there ICC service frames present?

 

  On "golden rule" in diagnosing DELPEN processes is to NOT attempt to STOP/ID the process again (but usually that advice is only ever given too late!).

A crucible of informative mistakes
Richard Brodie_1
Honored Contributor

Re: Troubleshooting DELPEN process

The program shouldn't have more than one ICC operation at once. It runs a fairly straightforward asynchronous request/response protocol. I couldn't see any sign of ICC operations on the stack, just stuff like:

EXE$EXIT_INT_C+002B0, NEXTASTMODE+0023C, THREAD_SWITCH_KPB_C+000F0, SYS$SYNCH_INT_C

 

This is what PWAIT makes of it:

 

Thread 0: state LEF AST pending SU active (none) blocked (none)
Process has been waiting for 00:00:02.81
Process thread resource wait is ENABLED
Process is marked for deletion
Process exec mode rundown is active
Analyzing process locks
        Process owns no locks
Event Flag Wait Mask EFWM 0000000D Wait Event Flag Cluster WEFC 4
Local Event flags 32-63 C0000000 31-0 C0000000
waiting for event flag 128 (EFN$C_ENF). EFWM is not relevant
process has 53 channels 0 of which are busy

Volker Halle
Honored Contributor

Re: Troubleshooting DELPEN process

Richard,

 

the key information is: 'Process exec mode rundown is active'

 

Is the process completely hung or is it still consuming CPU time ? Use SET TERM/WID=132, ANALYZE/SYS, set context to that process and report the output of SDA> CLUE CALL

 

Volker.

Richard Brodie_1
Honored Contributor

Re: Troubleshooting DELPEN process

SDA> clue call
Not available on I64  -  Use SHOW CALL or SHOW CALL/SUMMARY
SDA> show call
Cannot display call frame (error)
SDA>
SDA> show call/summary
%SDA-W-LOSTPROCESS, cleaning up: SDA's current process no longer exists; context now set to process running SDA

Volker Halle
Honored Contributor

Re: Troubleshooting DELPEN process

Richard,

 

does the process still consume CPU ?

 

Finding the exec mode rundown handlers for this process probably needs a thorough read through the IDSM (Internals and Data Structures Manual).

 

Consider to post the output of SDA> SHOW STACK and/or SDA> SHOW STACK/EXEC or attach a .TXT file with that output.

 

Volker.

 

Richard Brodie_1
Honored Contributor

Re: Troubleshooting DELPEN process

Volker, it doesn't consume CPU when you're not looking at it. When I'm poking around in SDA it clocks a few centiseconds.

 

Stack details attached.

Volker Halle
Honored Contributor

Re: Troubleshooting DELPEN process

Richard,

 

the exec mode run down handler routine addresses should be in the APLD and pointed to by ctl$gl_usrundwn_exec

 

What does SDA> exa @ctl$gl_usrundwn_exec;^d42*4  report (in the context of the hung process) ? If it reports any S0 space addresses, try to do an SDA> EXA/INS @value on each of them. These are the per-process exec mode rundown handlers.

 

Also what does SDA> EXA @EXE$GL_USRUNDWN_EXEC;10 report (system wide exec rundown handler vector) ?

 

Just trying to determine, which exec mode handler might have been called...

 

Good luck,

 

Volker.

Richard Brodie_1
Honored Contributor

Re: Troubleshooting DELPEN process

Volker, I think this is what you were asking for: it would be an S0 address, if sign-extended.

 

SDA> exa @ctl$gl_usrundwn_exec;^d42*4
00000000 00000000 00000000 0030E060  `à0.............     00000000.7FFB7CB0

Zeros suppressed from 00000000.7FFB7CC0 through 00000000.7FFB7D4F

0030E060 8EFA11D0 00000000 00000000  ........Ð.ú.`à0.     00000000.7FFB7D50
SDA> exa/ins @8EFA11D0
                        { .mii
SYS$IPC_SERVICES+67380:             alloc       r43 = ar.pfs, 10, 01, 00
                                    add         r9 = 200F68, r1
                                    add         r12 = 3FF0, r12

SDA> EXA @EXE$GL_USRUNDWN_EXEC;10

Virtual locations 00000000.00000000 through 00000000.0000000F are not accessible