topic Re: Process information from GET$JPI seems inconsistent in Operating System - OpenVMS

Process information from GET$JPI seems inconsistent

Nick Varley — Tue, 16 Dec 2008 10:02:09 GMT

We build and sell software that retrieves performance data from many operating systems (Windows UNIX Linux z/OS), including VMS systems. We are *not* VMS experts by a long shot, but we have built some software that retrieves useful information from VMS systems. We are however somewhat confused by the results of program that issues repeated GET$JPI calls to obtain details of running processes. We see a couple of odd things.

1. The same process apparently using less CPU in consecutive intervals, e.g. at 10:00 CPU=6957 at 10:15 CPU=4173, at 10:30 CPU=7132. We thought the CPU used values would be "gas meters", i.e. they only get bigger until the process ends. Calculating the CPU time used during an interval by a process results in negative values - and that can't be right.

2. Process IDs are duplicated - that seems odd too. I would have thought that every process in a given system would have a unique PID. We see that a process also has a start date/time and have used that provide a unique key, but even so, processes that we know are running don't appear to be present in every interval in the data we get back from GET$JPI.

Any ideas or pointers as to what is going on would be gratefully received.

Re: Process information from GET$JPI seems inconsistent

Heinz W Genhart — Tue, 16 Dec 2008 11:04:59 GMT

Hi Nick

first of all, welcome to the HP OpenVMS Forum.

Your problem sounds very strange.

PID's are absolutely unique within same cluster.

Also the problem you have with getjpi sounds impossible for me.

Could you supply some example code? Until now I can only assume a coding problem.

Regards

Geni

Re: Process information from GET$JPI seems inconsistent

Nick Varley — Tue, 16 Dec 2008 11:17:11 GMT

Thanks for the reply. I've attached some of the code.

Re: Process information from GET$JPI seems inconsistent

Nick Varley — Tue, 16 Dec 2008 11:18:06 GMT

...and also some output from the routine, for process ISEE$MAD. 3 different processes, all with different start dates and times, all with the same PID.

Re: Process information from GET$JPI seems inconsistent

Volker Halle — Tue, 16 Dec 2008 11:18:28 GMT

Nick,

welcome to the OpenVMS ITRC forum.

1. the used CPU time in a process is stored in the process header (PHD$L_CPUTIM) in 10 ms units and can only increment during the life of a process. You can look at the data in memory with:

$ ANAL/SYS
SDA> READ SYSDEF
SDA> SET PROC/IND=
SDA> EXA PHD+PHD$L_CPUTIM
SDA> EXIT

2. Process IDs are guaranteed to be unique within a system and even within a cluster. The fact that you don't get all the process data in each interval, may be caused by issues with the state of the process being queried, as some of the GETJPI calls may need to run in the context of that process and the state of that process may prevent such code to be executed immediately.

Volker.

Re: Process information from GET$JPI seems inconsistent

Heinz W Genhart — Tue, 16 Dec 2008 11:43:04 GMT

Hi Nick

for me it seems that this is basic code.
I dont have a basic compiler and I did not use basic the last 25 years.

That means that I can't help you.
But you can search some example code on google. Do a search for + openvms +basic + lib$getjpi and you will find a solution for your problem

Sorry for no better help
Regards

Geni

Re: Process information from GET$JPI seems inconsistent

Nick Varley — Tue, 16 Dec 2008 11:48:00 GMT

Thanks for the ideas so far. It has always been my thought that we're not doing it right, I'm just not sure where to go to find a better way. I have spent quite of lot of time already with my best friend Dr. Google...

Re: Process information from GET$JPI seems inconsistent

Jur van der Burg — Tue, 16 Dec 2008 11:59:15 GMT

Your program is buggy. You use sys$getjpi instead of sys$getjpiW which causes your program to continue while getjpi is still working on the data. And you specify an IOSB without bothering to check it....

So, use sys$getjpiw and check the both the return status AND the status in the iosb.

Jur.

Re: Process information from GET$JPI seems inconsistent

Volker Halle — Tue, 16 Dec 2008 12:02:06 GMT

Nick,

did you check, wheter a real process exists in the monitored system with a PID of 00153AB8 ? I bet this is some P0 space address in your program and not a real PID ! Just use $ SHOW SYSTEM, it will show you a list of the processes including their PID in column 1.

And note that the 3 different ISEE$MAD processes have only increasing values in their CPUTIM fields...

Volker.

Re: Process information from GET$JPI seems inconsistent

H_Bachner — Tue, 16 Dec 2008 12:09:01 GMT

Nick,

without digging deeper into the code (and installing a BASIC compiler) a few things come to mind when looking at your code:

- where do you construct the item list passed to $GETJPI?
- IOSB seems to be undefined/uninitialized
- you are using the asynchronous version of $GETJPI, but don't synchronize with an event flag and/or check the IOSB. You'd better use $GETJPIW in this case (unless you want to issue multiple calls in parallel - in which case I'm not sure how scanning the process list would work out).

This should give you a start to improve the code you posted. If more problems show up, get back here.

Good luck & regards,
Hans.

Re: Process information from GET$JPI seems inconsistent

Nick Varley — Tue, 16 Dec 2008 12:11:52 GMT

The code was just a snip, the items mentioned are properly initialised further up the program. I have re-posted the whole code should anyone be interested in seeing it all
.
I think the GET$JPIW route sounds the most hopeful, and I will talk to our developers to create version that uses that routine rather than GET$JPI.

Thanks again to all who have so far responded.

Re: Process information from GET$JPI seems inconsistent

Joseph Huber_1 — Tue, 16 Dec 2008 12:36:33 GMT

In addition to the synchronization issue, some of the data from the process may be missing/not updated due to the
JPI$M_NO_TARGET_INSWAP
flag (which itself of course is a good thing).
See the system services manual for details.

Re: Process information from GET$JPI seems inconsistent

Hein van den Heuvel — Tue, 16 Dec 2008 12:42:46 GMT

I'm sure that GETJPIW solved the coding problem, but just in case it does not...

$GETJPI works. Period. It has been working for 30 years and no 'rookie' OpenVMS user is going to find a glaring bug in it today.

So if someone writes "Process information from GET$JPI seems inconsistent" then I read that as: "we have created a broken program and woudl like help figuring out what we did wrong".

Using that as a starting point and focussing on "what could we possibly have done wrong" is more productive than speculating as to whether a core OpenVMS system call might have gone wrong on you.

If you still have a problem, then consider adding a LIB$SPAWN call to the program (outside the loop) to perform a SHOW SYSTEM into a file at the same time the program.

The output should be 95% the same as the test program output file. Any and all difference might help explain what is going wrong.

In fact "SHOW SYSTEM" and the proposed program are so close that I wonder why the program was even written?! The only significant difference is "IO" vs "DIRIO" and I wonder whether it is fully understood what the difference between those tow is (BUFIO) and which would be more appropriate.

Now IF you opt to waste development cycles on rolling your own SHOW SYSTEM, then consider adding USER/EXEC/KERNEL cputim, as that is a missing feature from SHOW SYSTEM and can help a lot when trying to understand system behaviour.

Cheers,
hein.

Re: Process information from GET$JPI seems inconsistent

Nick Varley — Tue, 16 Dec 2008 12:47:59 GMT

Thank you Hein. Be assured, I am under no misconception that we have done this right and VMS is wrong - I'm a z/OS mainframe guy by breeding (some 30 years now), so I never assume that anything that appears to be wrong is the fault of the operating system.

Your idea is really good and will help us to cross-reference our results with the official version. We have our own deep dark reasons for doing it this way which I won't bore you with right now.

Re: Process information from GET$JPI seems inconsistent

H_Bachner — Tue, 16 Dec 2008 14:20:51 GMT

Hello Hein,

> In fact "SHOW SYSTEM" and the proposed program are so close
> that I wonder why the program was even written?!

in his first posting, Nick mentioned that the goal was to collect performance data from various operating systems. This sounds like issuing DCL commands and looking at the result is not an option, rather than getting the information in machine readable format.

If my assumption is correct, I think going the $GETJPI(W) way is far superiour to scanning the output of a DCL command - which may change across versions, and with uncertain behaviour if some field overflows the allocated width or intended value range.

Remember SHOW SYSTEM showing delta times instead of used CPU time? Guy Peleg fixed this a couple of years ago for "SETI at Home" :-)

Cheers,
Hans.

Re: Process information from GET$JPI seems inconsistent

Richard Brodie_1 — Tue, 16 Dec 2008 15:05:24 GMT

"did you check, wheter a real process exists in the monitored system with a PID of 00153AB8 ? I bet this is some P0 space address in your program and not a real PID !"

On a quick look, I vote for Volker being correct, and you are using f$faol incorrectly; I suggest you check the unformatted values.