topic Re: Suspended process cannot be removed in Operating System - OpenVMS

Suspended process cannot be removed

Willem Grooters — Fri, 26 Nov 2004 09:22:28 GMT

A system manager encountered a problem with a job that cannot removed.

The process has been started in batch, running BACKUP, so allocated and accessed the tape unit. Now it is in LEF state for a long time - and no tape activity has been noticed for this period, where expected.

He tried:
- STOP/ID: didn't work - the process is still present.
- STOP/ABORT and then deletion and recreation of the batchqueue didn't work - the process is still present.

SHO PROCESS/ALL tells me the process is suspended, so I cannot find out what image it's running (if any. It must have been backup.exe but I'm not sure whether this has finished). So I turned to SDA, that showed the channel to the tape drive is still busy - see attached SDA output.
It seems no files opened on DKA0: SHO DEV/FILES of DKA0 shows no files opened by this process.

I also checked whether it is requesting another tape: the Backup command used does not contain a /NOASSSIST qualifier so it would show if so, but REPLY/STATUS does not show such a request.
Could be backup?
$ install list backup/full

DISK$DKA0:.EXE
BACKUP;1
Entry access count = 5

Since the process keeps the tape unit, I need to get rid of it before the next backup job starts. Is there a way to achive that without having to reboot the machine?

Re: Suspended process cannot be removed

Lokesh_2 — Fri, 26 Nov 2004 09:27:57 GMT

Hi William,

How the tape drive is attached to the system? Probably a restart (power recycle) of the tape drive can resolve this issue.

Thanks & regards,
Lokesh

Re: Suspended process cannot be removed

Ian Miller. — Fri, 26 Nov 2004 09:34:28 GMT

the process is marked as delete pending due to the STOP/ID which probably explains the process is suspended message. Lokesh suggestion of power cycle the drive may allow the I/O to be cancelled and the process to end.

Re: Suspended process cannot be removed

Willem Grooters — Fri, 26 Nov 2004 10:06:35 GMT

...power cycle the drive may allow the I/O to be cancelled and the process to end...

So we did, but it didn't work. Even unloading the tape didn't help.

I checked the device - see attachment. It seems to be kept busy (note that this has been made after power cycle).

Willem

Re: Suspended process cannot be removed

Ian Miller. — Fri, 26 Nov 2004 10:16:40 GMT

I see mntverip in the status. I wonder if the mount verify could be encouraged to end. There was a command at the IPC> prompt but I wonder if there is a modern alternative.

Re: Suspended process cannot be removed

Kris Clippeleyr — Fri, 26 Nov 2004 10:21:27 GMT

Willem,

As far as I can see, the process is waiting for a buffered I/O to complete
(Buffered I/O count/limit 149/150) that will set local event flag number 31
(Waiting EF cluster 0 Event flag wait mask 7FFFFFFF)
If you could force that event flag, maybe than the process will be willing to commit suicide. I'll check if I have some dirty program at hand here. Otherwise, it's reboot I'm afraid. Btw, what version of VMS? (or did I miss that?)

Greetz,
Kris

Re: Suspended process cannot be removed

Ian Miller. — Fri, 26 Nov 2004 10:28:17 GMT

the process IRP waiting behind the mount verify IRP. Try setting the dynamic system parameter TAPE_MVTIMEOUT to 1

Re: Suspended process cannot be removed

Volker Halle — Fri, 26 Nov 2004 10:30:36 GMT

Willem,

looks like some IO is stuck in the driver someplace. You could try IPC> C MKC600: to cancel mount-verification, but it's risky and I won't have much hope that it will clean up the sitaution.

But isn't there the 'old trick' of changing the SCSI id of the tape and bring it back as a new device this way ?

Volker.

Re: Suspended process cannot be removed

Ian Miller. — Fri, 26 Nov 2004 10:33:41 GMT

Volker (or anyone else that knows)
Is there a modern equivilent to
^P
>>> DEP/I 14 C
>>> CONT
IPC> C $1$MUA0:
IPC> ^Z

I know that AMDS can do the same as
^P
>>> DEP/I 14 C
>>> CONT
IPC> Q
IPC> ^Z
and was just wondering.

Re: Suspended process cannot be removed

Willem Grooters — Fri, 26 Nov 2004 10:34:14 GMT

Ian,
I noticed that as well and it wonders me. Mount Verification In Progress - on a TAPE drive. VMS's sense of humour, I guess.
Kris,
VMS 7.3-1, unsure of ECO level (but WAY behind, knowing the site). I would like to see that program - if you you can find it (you know where to reach me)

Anyway:
It was short notice, I also have to test some other things in startup and sysman wants the backup to be done tonight. So I have rebooted the box (It's been a busy week...)
All suggestions and explanations will be kept in mind for the next time, which will, no doubt, be somewhere next week.

Have a good weekend.

Willem

Re: Suspended process cannot be removed

Ian Miller. — Fri, 26 Nov 2004 10:43:53 GMT

I had a program for VAX/VMS to set local event flags in other processes
http://vms.process.com/scripts/fileserv/fileserv.com?ESET
but never did update it for AlphaVMS. In the case of an I/O the I/O status block is checked after the event flag is set. If the i/o status block is still set to 0 SS$_PENDING then the process goes back to sleep.
I think setting TAPE_MVTIMEOUT to 1 can lead to the mount verify ending after a few mins.

Re: Suspended process cannot be removed

Willem Grooters — Fri, 26 Nov 2004 10:49:43 GMT

Volker, Ian,
IPC means "console access" doesn't it? Anyway I see in Ian's reply ^P, so yes.
To change the ID of the tape drive would have been a possibility.
But in either case, I would need to get into the computerroom. But SYSMGR went home and let me decide what to do (I LIKE THE GUY!).

(Even if he were there, the IPC trick wouldn't work anyway. There is no console on this machine; management is all done using telnet sessions...)

Re: Suspended process cannot be removed

Volker Halle — Fri, 26 Nov 2004 10:58:30 GMT

Willem,

if you had forced crashed the node instead of a simple reboot, there would have been a chance to look at the dump, but...

Ian, Kris,

just setting the Event Flag would not have been enough. There are also bits in the CCB which need to be adjusted and the JIB and so on...

DECamds does not seem to offer a 'cancel MV' function (on the Fix Options Page). Can't think of any other tool.

Have a nice weekend,

Volker.

Re: Suspended process cannot be removed

Uwe Zessin — Fri, 26 Nov 2004 11:27:32 GMT

Willem,
mount verification on a tape device - that is not supposed to be funny.

It helps in those cases when one removed a tape media and put in a different one although the former one was logically mounted - works the same way with removable disk media (anybody remember RM03s, RM05s, the RP family and so on?). It is one more use of the mount verification mechanism.

Re: Suspended process cannot be removed

Jan van den Ende — Fri, 26 Nov 2004 14:58:07 GMT

Willem,

I am with Uwe here: forcing MV timeout HAS helped us on different occasions (but, IIRC, seems to have been all way back in Vax times, don't know if Alpha is different).
Also:
Volker's suggestion of unhooking tape device, and switching Device ID, followed by MC SYSMAN IO A has helped us out more than once, and that definitely WAS alpha. Then simply let the process slumber in limbo till there is some other reason for reboot.

fwiw,

Cheers.

Have one on me.

jpe

Re: Suspended process cannot be removed

Ian Miller. — Fri, 26 Nov 2004 16:43:44 GMT

I would be interested to know if I think setting TAPE_MVTIMEOUT to 1 causes the end of mount verify sooner in recent versions of VMS as it used to.

Re: Suspended process cannot be removed

Uwe Zessin — Sat, 27 Nov 2004 02:45:38 GMT

Ian,
if I recall correctly, this parameter is read when the volume is being mounted - you cannot affect a currently mounted volume just by lowering the parameter.

Re: Suspended process cannot be removed

Volker Halle — Sat, 27 Nov 2004 10:12:28 GMT

Ian,

a nice question for the weekend ;-)

When mount verification starts on a device, the current absolute time EXE$GL_ABSTIM (in seconds) plus the value of *MVTIMEOUT is stored in the MVIRP (mount verification IRP) at IRP$L_MV_TMO.

So dynamically decreasing TAPE_MVTIMEOUT would NOT help, if the tape already is undergoing mount verification.

Only chance is to locate the MVIRP and put a 0 in IRP$L_MV_TMO - needs a little hacking with DELTA. Locating the MVIRP is easy (see Willem's second attachment).

Canceling mount-verification should be tried with DISM/ABORT or the IPC> C MKcn: command.

Volker.

Re: Suspended process cannot be removed

Ian Miller. — Sat, 27 Nov 2004 10:59:14 GMT

I thought I had read somewhere that altering the value of TAPE_MVTIMEOUT could affect mount-verifies inprogress - I guess not.

If I put 0 into IRP$L_MV_TMO when does this get checked - is there a reguular timed event that checks this?

Re: Suspended process cannot be removed

Volker Halle — Sat, 27 Nov 2004 11:36:34 GMT

Ian,

mount verification is tried in a loop (with some little pauses inbetween the IO$_PACKACK calls), until

- it succeeds
- it's canceled (by DISM/ABORT clearing the VCB$V_MOUNTVER bit)
- it times out.

Volker.