topic Re: "Hang" on return in Operating System - OpenVMS

"Hang" on return

Willem Grooters — Fri, 13 Mar 2009 11:06:09 GMT

VMS 7.3-2, Alpha
Login as TMG1_WILLEM (low-priv user), execute a command:

$ SPAWN PIPE SETP tomg2 ; ; SETENV TOMG1

SETENV is a procedure that changes the process and job environment to fit a particular environemnt, different from login: UIC, default, logicals, process rights and privileges (both AUTHORIZED and DEFAULT), using two (privileged) images and DCL. The user does have an account in this environment (TMG2_WILLEM) but can not normally log in directly into this environment.
is executed in the context of user TMG2_willem - with elevated privileges.

The source of the original images is lost, and the functionality has been reverse-engineered from the original, by examining what it does to process end job environment.
In most cases, it seems to work fine. Changes in the process environment can be observed in the attachment.

However, above code sequence shows that there is still a difference to be handled.
Using the old images, it doesn't matter if fails causing an $ EXIT; the subprocess will end and control will be returned to the main process (though the changes in the job environment will still be there). The new images however cause the main process to hang, and the process needs to be killed (^C nor ^Y have any effect).

It seems to me that the subprocess does not signal the main process of it's termination.

I also tried (to check process and devices:

$ SPAWN PIPE SETP TOMG2 ; ANA/SYS

and this also causes the main process to hang when SDA is exited, no matter how, when the new images are used.

My thought is there must be some protection that needs to be altered, or an ACE added, (like I did on TT:) to signal the end of the subprocess, but I couldn't locate anything.

Any idea on how to find out what device may be inaccessible?

Re: "Hang" on return

Hein van den Heuvel — Fri, 13 Mar 2009 11:58:50 GMT

No immediate help, just a couple of question which may help others.

First, roughly, what does the new code do? Straight system service calls (impersonate!) or kernel mode hackery twidlling bits in Process control blocks, process headers?

> TMG1_WILLEM (low-priv user),

It looks to me that this username has all the priviliges in the world: CMKRNL (useful to grab any other priv, and to issue SET UIC), and WORLD and so on. Not low-priv and does nto need an installed helper.

>> $ SPAWN PIPE

What is the purpose of the SPAWN before the pipe?

Are there DETACHED processes, or just changed to the sub-processed.

I suspect that one of the processes is waiting for a MAILBOX message. Use ANALYZE/SYSTEM and SHOW PROC/CHAN looking for 'busy'. You may also want to use the SDA MBX extention.

Hope this helps a little,

Groetjes,
Hein.

Re: "Hang" on return

Wim Van den Wyngaert — Fri, 13 Mar 2009 13:09:17 GMT

Does it stop when you wait about 10 minutes ?

Does it use T2T ? If the receiver does an exit without reading/closing sys$net it will receive a wait of 5 minutes (was on VMS 6.2).

Wim

Re: "Hang" on return

Willem Grooters — Fri, 13 Mar 2009 13:19:08 GMT

Actions of the new code: just what the old image seems to be doing (according SHOW PROCESS/ALL output and other information gathered), but basically as described in the attachment. No impersonation, plain system calls and direct access into system structures as well.
The images in the procedure are installed with the privs required, so any user can run the procedures, without elevated privs.
on SPAWN PIPE....The original sequence I got changes environment, copies a file in that environment that is otherwise inaccessible, changes ownership of this copy, restores the original environment and exits - as a single entity. There are other ways to achieve this but this is the code used in the production environment.
On mailbox: I guess so: My idea is that writing a message to a termination mailbox fails because the process' UIC has changed. However, I just noticed that the original program may change the process-UIC as well - but probably just in the process context, not in the JIB (as the new code does).

Re: "Hang" on return

Willem Grooters — Fri, 13 Mar 2009 13:24:33 GMT

Wim: hasn't been mentioned; worth giving it a try (but I think it won't stop)
No T2T - what's T2T? - AFAIK.

Re: "Hang" on return

Willem Grooters — Fri, 13 Mar 2009 13:26:01 GMT

T2T + SYS$NET = task to task. Could have thought about that: No, not involved.

Re: "Hang" on return

Willem Grooters — Fri, 13 Mar 2009 14:00:42 GMT

Wim; waited half an hour but still no reaction: subprocess is stopped, main process remains in HIB, no open channels.

I looked into ACC for the last attempt ($ SPAWN PIPE SETENV TOMG2 ; ANA/SYS), that I stopped with ^Z (normal exit). It didn't return either, this is what ACC returns on this proces:

SUBPROCESS Process Termination
------------------------------
Username: TMG1_WILLEM UIC: [TMG2_P,TMG2_WILLEM]
Account: TMG1_P Finish time: 13-MAR-2009 15:06:17.30
Process ID: 20200155 Start time: 13-MAR-2009 15:06:12.64
Owner ID: 20200154 Elapsed time: 0 00:00:04.65
Terminal name: Processor time: 0 00:00:03.76
Remote node addr: Priority: 4
Remote node name: Privilege <31-00>: 521DB02D
Remote ID: Privilege <63-32>: 00000000
Remote full name:
Posix UID: -2 Posix GID: -2 (%XFFFFFFFE)
Queue entry: Final status code: 00F48009
Queue name:
Job name:
Final status text:
Page faults: 479 Direct IO: 101
Page fault reads: 87 Buffered IO: 183
Peak working set: 5632 Volumes mounted: 0
Peak page file: 172912 Images executed: 7

Re: "Hang" on return

Hein van den Heuvel — Fri, 13 Mar 2009 14:53:37 GMT

The status is just the ANALYZE/SYSTEM you used:

$ write sys$output f$mess(%x00F48009)
%SDA-S-NOMSG, Message number 00F48009

The HIBER state is interesting.
It suggests it is NOT having a (mailbox) IO outstanding, but it may have an attention AST ready to fire. So I woudl still check for the channels and such.

Have just tried whether a simple $WAKE call will make it happy?

Does the tool need to have a SYS$WAKE call, to add a 'pending' wake, or use SYS$SCHDWK 'just in case'?

Hein.

Re: "Hang" on return

Hein van den Heuvel — Fri, 13 Mar 2009 15:07:25 GMT

fwiw...

There is (oddly?) no standard tool to do SET PROC/WAKE/ID=...

Here is a trivial 'wake' tool.

$ type wake.c
#include
int pid, sys$wake();
main (int argc, char *argv[])
{
sscanf( argv[1], "%X", &pid);
return sys$wake(&pid,0);
}
$

And here is that tool in a test:

$ type hiber.c
#include
#include
int sys$hiber();
main ()
{
printf ("Hiber for %08X\n", getpid());
sys$hiber();
printf ("Goodbye.\n");
}
$
$ spawn/nowait run hiber
%DCL-S-SPAWNED, process HEIN_7111 spawned
$
Hiber for 00000497
$
$ mcr sys$login:wake 497
Goodbye.
$

Goodbye,
Hein.

Re: "Hang" on return

Jan van den Ende — Fri, 13 Mar 2009 15:12:02 GMT

Willem,

is this SETENV the same as the one used in the environment we both have in our history?
If YES, then you probable should contact Oswald.
I no longer have the direct contact info, but I think you will know where to find him.

hth

Proost.

Have one on me.

jpe

Re: "Hang" on return

John Gillings — Mon, 16 Mar 2009 02:08:14 GMT

Willem,

Rather than trying to plug all the inevitable holes in your SETENV utility (it will be a never ending task!), why not do this using existing, working and supported mechanisms?

First thing that springs to mind is:

$ SUBMIT/USER=TOMG2
$ SYNCHRONIZE/ENTRY='$ENTRY'
$ stat=$STATUS

Yes, SUBMIT/USER is a privileged command, but you have the required privileges.

There are many other possibilities.

Re: "Hang" on return

Willem Grooters — Mon, 16 Mar 2009 06:48:20 GMT

Hein: There is no SET PROCESS/HIBER either, so I'm not surprised ;)
Jan: No, entirely different
John: this might be a better solution _in this case_ but it's not always applicable. The same sequence can be executed by non-privileged users in which case SUBMIT/USER is not possible. Besides - the method has been in use over 10 years and is embedded in quite a few (production) scripts....

Re: "Hang" on return

Wim Van den Wyngaert — Mon, 16 Mar 2009 06:57:11 GMT

set mes sys$message:prgdevmsg.exe
write sys$output f$mes(%xf48009)
%SDA-S-SUCCESS, success

fwiw

Wim

Re: "Hang" on return

Willem Grooters — Mon, 16 Mar 2009 12:35:26 GMT

Wim: Obvious :). The problem is not SDA or any command or script executed when SETENV is run, but what happens id the subprocess is stopped (no matter what way). Normal exit of SDA will fail to wakeup the main process when the new exe is used, where the old one works fine.

I've done some more investigation with two processes; one using the old image, the other using the new. In both; I executed SPAWN and SETENV; then, I analyzed the system and rercorded JIB, PCB and PHD, and compared the outcome (just the labels and values; addresses removed. The result v=can be found in the attachement. Most differences are obvious, but I do have one that I could not explain; that one is a few lines below PCB$L_INITIAL_KTB in the form,marked "******"in the center.

Could it be that some P1 data needs to be altered? If so,what would I need to look at?

Re: "Hang" on return

Hoff — Mon, 16 Mar 2009 13:04:46 GMT

Among other wrinkles, I've seen all manner of ownership and logical name table and mailbox issues when SPAWN is involve such as often arises within a PIPE.

DCL does not expect processes to swap identities on the fly.

SET UIC (deprecated) regularly blows up processing with DCL, for instance.

Spoofing code also tends to break at upgrades.

And some of the spoofing code I've reviewed has introduced more and larger security holes than its authors had intended to plug.

Find and use another way to reach your goals here. Create a server and pass command(s) into it. Or better (since passing in user commands is a path for injection) pass in requests or codes using a fixed and known grammar.

And yes, I know you'll ignore the comments here. Why? Well, I ignored the folks that tried to tell me this, too. A design that allows spoofing the user security context on the fly in an interactive or command-level environment is generally somewhere between a Really Bad Idea and a Massively Bad Idea. (qv: XSS, SQLI, etc.) Simplify the design. Mailboxes and hangs are the least of the issues.

Put another way, I'd suggest a redesign here. It'll be easier to support, easier to maintain, easier to lock down, and the results will survive upgrades.

Re: "Hang" on return

Willem Grooters — Mon, 16 Mar 2009 14:31:08 GMT

Hoff: I fully agree that redesign (of the whole environment) is the right way to go and if I had the chance of doing so, I would follow that direction. But I'm not in charge in planning or architecture of this project and just have to get things working, to meet the requirement that 'existing code should work unchanged'. No doubt that sounds familiar :( .

We're well on the way for that:
Without SPAWN, it's fine.
With SPAWN, it's fine once the situation is set back to normal. But if the subprocess finishes before that, the main process is not notified. There must be something within VMS that causes this to happen; the only question is: What.

(Once I know that, the program could be altered to work properly. That will be the last change made to the code - further maintenance is not foreseen.)

Re: "Hang" on return

Jess Goodman — Mon, 16 Mar 2009 16:44:55 GMT

Your problem is because after your SETENV the sub-process no longer has write access to its termination mailbox. The parent process is waiting with a write-attention AST on this mailbox and will not continue until a process termination accounting record is written there.

This is easy to demonstrate:
$ SPAWN
%DCL-S-SPAWNED, process GOODMAN_55891 spawned
%DCL-S-ATTACHED, terminal now attached to process GOODMAN_55891

$ SET UIC [100,1]
$ SET PROCESS/PRIV=NOBYPASS
$ LOG
Process GOODMAN_55891 logged out at 16-MAR-2009 17:37:05

(parent process hangs)
--------------------------------------------
$ SPAWN
%DCL-S-SPAWNED, process GOODMAN_13828 spawned
%DCL-S-ATTACHED, terminal now attached to process GOODMAN_13828
$ SAY F$GETJPI(0,"TMBU")
3078
$ SHOW DEVICE/FUL MBA3078:

Device MBA3078:, device type local memory mailbox, is online, record-oriented
device, shareable, mailbox device.

Error count 0 Operations completed 1
Owner process "" Owner UIC [STAFF,GOODMAN]
Owner process ID 00000000 Dev Prot S,O:RWPL,G,W
Reference count 1 Default buffer size 256

$ SET SECURITY /CLASS=DEVICE MBA3078: /PROTECTION=W:RWED
$ SET UIC [100,1]
$ SET PROCESS/PRIV=NOBYPASS
$ LOG
Process GOODMAN_13828 logged out at 16-MAR-2009 17:43:11
%DCL-S-RETURNED, control returned to process GOODMAN
$
(parent process resumes)

Re: "Hang" on return

Hein van den Heuvel — Mon, 16 Mar 2009 17:00:00 GMT

Hi Jess,

Thanks for confirming what several suspected.
Nice & easy reproducer. So the solution for Wim will be to add code to change the process termination mailbox protection or owner.

Hein.

JPI$_TMBU
Returns the termination mailbox unit number, which is a longword integer value.

Re: "Hang" on return

John Gillings — Mon, 16 Mar 2009 21:09:00 GMT

Willem,

(not sure why I bother answering your questions when you invariably ignore the advice, but anyway...)

OK, so the problem is really that your procedure exits abnormally, preventing your SETENV from reverting to the old environment, so it can't write to the mailbox etc...

Rather than trying to fix the flawed mechanism, how about trying to ensure the reversion happens regardless of what the procedure does? Since I don't know the nature of what it's doing, or how its failing, here are a few suggestions:

$ SPAWN PIPE SETP tomg2 ; SET NOON ; ; SETENV TOMG1

$ SPAWN PIPE SETP tomg2 ; SPAWN @procedure ; SETENV TOMG1

Failing that, you could use a big hammer and fix the protection on the mailbox prior to fiddling the UIC. Of course, this opens a potential security hole of sorts, as anyone will be able to write to the mailbox

$ SPAWN PIPE SET SECURITY/CLASS=DEVICE/PROTECTION=(W:RW) DCL$ATTACH_'F$GETJPI("","PID")' ; SETP tomg2 ; ; SETENV TOMG1

Note that the F$GETJPI executes in the context of the parent process, so it gets the correct mailbox name.

To plug the security hole you could insert an ACE granting access only to the user to which you're going to SETP, rather than the blunt instrument "W:RW". Once that's done, you may even be able to eliminate the SETENV to revert.

Re: "Hang" on return

Willem Grooters — Tue, 17 Mar 2009 07:01:42 GMT

Jess: I already suspected that, just the point how to get that device ....
Hein: That will do for that; I already add an ACE to the terminal, I can easily add another device :)
John: See my comment on Hoff. I can't change things here.
Your last suggestion on how to change the ACL on the termination mailbox is what I will insert.