Find out who has a particular file open in i/o mode

[Homer Shoemaker]

OpenVMS V8.3 on node P2660 13-JUL-2010 09:27:25.53 Uptime 404 12:21:32
System: P2660, HP rx2660 (1.59GHz/9.0MB)
HP Cobol compiler version 2.9

Problem: Find out who has a particular file open in i/o mode.

When we run the program that checks for locked files prior to running an AP post using the code below we get our message telling us the file is in use. And that's fine.

MOVE "VENDOR" TO FILE-NAME.
OPEN I-O VENDOR ALLOWING READERS.
IF NOT SUCCESSFUL
PERFORM 900-DISPLAY-MESS-BEEP
MOVE 6 TO JCL-CODE
GO TO 999-TIE-UP.
CLOSE VENDOR.

There are many people running programs that could have opened the file.

What I have tried:
(1)
$ INQUIRE/NOPUNCTUATION FIL
$ FILNAM = F$SEARCH("''FIL':")
$ DRV = F$PARSE("''FILNAM'",,,"DEVICE")
$ SHOW DEV/FILES/NOSYS/OUT=FS1.TMP 'DRV'
$ SEA FS1.TMP 'FIL'

I may see thirty users, many with multiple sessions, with processess using that file. I cannot call every one of them and ask them to please back out of the program that they're using just to see if they're the one who has the file opened in i/o mode. It takes too long and I'm interupting too many people.

(2)
$ MYPID = F$GETJPI("","PID")
$ MYUIC = F$GETJPI("","UIC")
$ MYGRP = F$GETJPI("","GRP")
$ NEXT_ID = ""
$!
$GETID:
$ PID = F$PID(NEXT_ID)
$ IF PID .EQS. "" THEN GOTO FINISH
$!
$ UIC = F$GETJPI(PID,"UIC")
$ GRP = F$GETJPI(PID,"GRP")
$ INM = F$GETJPI(PID,"IMAGNAME")
$ SNM = F$PARSE(INM,,,"NAME")
$!
$ WRITE SYS$OUTPUT UIC," ",SNM
$ GOTO GETID

I can also get a list of all the images in use by all the process on the system. And then search the list for images that I know can open the file in i/o mode. The problem is that many of them can, depending on parameters passed to them at program startup, or switches in another file defining user permissions, or which features of the program are being used. Consequently, this doesn't narrow my search much at all. There are many people executing images that could have opened the file.

(3)
I looked into using files_info from the freeware disk. But I got in over my head quickly trying to get it to work on Integrity. I don't think the macro's are going to work.


Anybody have any ideas?

[P Muralidhar Kini]

Hi Homer,

>> Find out who has a particular file open in i/o mode.
By I/O mode, do you mean to find out if
- any process doing a write IO to the file currently (Read IO is ok) ??
or
- any process is doing a Read/Write operation to the file currently

Regards,
Murali

[Homer Shoemaker]

I mean, who has opened the file so that I cannot open it ALLOWING READERS only.

[Homer Shoemaker]

We use the COBOL OPEN statement, and the files that I am talking about are existing and indexed, so that we usually only open them using OPEN INPUT or OPEN I-O. So when I referred to i/o, I was referring to the COBOL syntax. Sorry for the confusion.

Can you help with the problem?

[Hein van den Heuvel]

>> Anybody have any ideas?

I have an idea, but not a finished solution, and no time just now to finish it.

You can perhaps use the attached source for a program which display RMS record lock holders.

This is a relatively new, not yet published tool precisely because I wanted to add file lock handling. It is in production for record locks.

The first step of the idea is to simply construct an RMS$ file lock resource name, get a (NL) lock and use GETLKI to query that lock for details on other holders.

The second step is to convert the lock, asynchronously to a blocking situation (because of those IO users) and NOT to wait, but to use GETLKI with the LKI$_BLOCKEDBY option to directly get a list of blockers.

That's what the program does for record locks already, but not yet for file locks.
Look for the comment 'qqq' or text: 'implemented'. SMOP!

Oh, and you'll need CMEXEC for the RMS lock, and CMKRNL is we need the XQL lock for the file of course.

Hope this helps some,
Hein

[Steven Schweda]

Perhaps:

help show devices /files

[Homer Shoemaker]

Hein:

SMOP! Indeed! LOL. I am going to get an education just following what you have already done. Let alone adding to it. Thank you very much. If I succeed I will post it here.

Steven:

Thank you for your response. I did try that. See (1) above for the problems that I ran into. I do value the generous contributions of time and effort from the experts and colleagues that I find here. Is there something else I missed?

[Verne Britton]

Brute force ??

Assuming you have all the source code, search everything to find the programs that do not play nice with others :-)


Verne

[Homer Shoemaker]

Verne:

Thank you for your reply. Well, that's certainly a possibility. But I think quite involved. And though it may come to that, I'm going to hold off for now.

It's not that the other programs don't play nice, it's just that they necessarily restrict the way the file can be opened by a subsequent program. I cannot just prevent end users from doing their jobs. (Then I wouldn't have a job!) And since the same program can open a file different ways depending on what the user wants to do, I would have to create a way to track which users have opened the file in a certain manner. Maintaining that list could be troublesome. But I think the OS is already doing that anyway, and probably better than I could.

[John Gillings]

Homer,

I'm not sure I understand this:

"checks for locked files prior to running an AP post"

What's the reason for checking, and what do you want to do with the process(es) when they're found?

Maybe instead of building complex privileged programs which dig into OpenVMS data structures to find PIDs, then deal with all the nasty timing issues you WILL encounter at the worst possible time (consider what if someone starts another program after you've scanned, but before whatever it is you want to do with exclusive access?), perhaps consider designing a mechanism by which your programs can cooperate in how they access the file.

Thinking on the fly, maybe something using (non-privileged) locks and blocking ASTs to signal processes that they should relinquish write access to the file?

Another option might be to have processes that open key files for write access put a flag in a shareable logical name table (LNM$JOB?) noting their PID. Program the equivalent of

$ DEFINE/JOB RW_ 'F$GETJPI("","PID")

You could then find the processes with:

$ SHOW LOGICAL/TABLE=* RW_

[Phil.Howell]

From decus/freeware there are some other options.
One that I used ages ago, but may only work on vax is findlock from Mr. Lomasky http://www.digiater.nl/openvms/decus/vax90a/lomasky/

Also on more recent freeware - DBS-SCANLOCKS
http://www.digiater.nl/openvms/freeware/v80/dbs-scanlocks/

and findlocks
http://www.digiater.nl/openvms/freeware/v50/findlocks/

[Richard J Maher]

Hi Homer,

As usual, with anything RMS (most things really) I'd go with Hein's advice.

Having said that there would be easier code (and in better languages) just to emulate the RMS file lock (DVI$_DEVLOCKNAM + FID in EXEC mode IIRC) and then do a $GETLKI to see who's blocking you.

If no one else has a simple example from one of the freeware tapes and/or you'd like and example in the language you're already using then let me know.

Cheers Richard.

PS. I thought there were some good (similar to RDB's rmu/show locks/mode=blocking or culprit etc in freeware somewhere?)

PPS. Bad luck to the Oranje :-)

[Robert Brooks_1]

Having said that there would be easier code (and in better languages) just to emulate the RMS file lock (DVI$_DEVLOCKNAM + FID in EXEC mode IIRC) and then do a $GETLKI to see who's blocking you.

--

Yeah, good luck! dvi$_devlocknam returns a string that's perhaps not what one would expect!

From EISNER::

$ write sys$output f$getdvi( "$1$dga1:", "devlocknam" )
000000202020205359534148504C4102

I tried to fix it, but was rebuffed under the "we don't want to change existing behaviour" reason.

If you "dehexify" the string, you should be able to feed it to $getlki . . .

-- Rob

[Richard J Maher]

Rob,

I think you'll find that it's only a nicety of the F$GETDVI lexical to convert it to a hexidecimal string on output and that sys$getdvi returns a lovely 16 byte dvi$_locknam as usual. And the volume lock living in bytes from 2 for 12.

But then I'm sure someone that works with VMS i/o would know that, so I'm going to write a little tester toi make sure.

I've been in bed sick most of the day and I've just found an old program that suggests in the strongest possible terms that the RMS resource name for file is "RMS$" + FID + DEVLOCKNAM at exec mode.

I'll give it a go and report back.

Cheers Richard Maher

[Richard J Maher]

No need, just looked at Hein's code and that is in fact the case.

[abrsvc]

While the utilities already discussed may get you moveing in the short term, perhaps a different avenue should be followed for the long term.

A similar issue arose with software that i currently support as well. The solution here was to create and use a lock on the file as a resource. Each process that takes out the lock can be seen in SDA by using the SHOW LOCK/NAME="x" command. If you take out the lock in CW mode when updating the file, the list should be easily seen. This would require a call to the ENQ system service along with the open statement. Also, if your program needs exclusive access, the lock mode would be EX for you and that can trigger an AST for the other programs to take action. That action might be to close the file until awakened by another AST sent by you.

Yes it will take program modifications, but if implemented properly, will renmove any manual intervention in the future.

Dan

[Richard J Maher]

Hi Hein,

If I follow you correctly you're saying that the RMS$+FID+DEVLOCKNAM is simply a RMS place holder parent resource for child record locks and does not serve a meaningful purpose with regard to file-locking?

If I've misunderstood please let me know.

If the XQP is where all the magic lies can you please detail the DLM reasource names reqd to successfully maintain a file lock?

Ok, so as bad luck would have it, it's all at kernel mode and I guess we're still talking FIDs and DEVLOCKNAM(VOLNAMs)

From my lovely DIR_WATCH utility (for checking when a file is created) I am comfortable with the "F11B$v" + volname as the parent followed by "F11B$s" + DID for the child as the serialization lock but, if I want to place a lock on a particular file, what is the "x" in "F11B$x", is it a child of the volume lock, and am I correct in presuming the rest of the resource name is just the FID?

Cheers Richard Maher

PS. Still coughing up oysters so might have time to do it tomorrow :-)

[Hoff]

Others have better addressed the immediate question.

Here's an overview of locking and a quick review of the groups of locks used by RMS, and some example code:

http://labs.hoffmanlabs.com/node/492

http://www.eight-cubed.com/examples/framework.php?file=sys_cmexec.c

AFAIK, the format of and details of the RMS locks are not officially documented. As for available documentation resources, have a look at Kirby McCoy's VMS File Systems book, and look in the appendices of most any "recent" VMS Internals and Data Structures (IDSM) book.

ANALYZE /SYSTEM (SDA) in recent releases has good lock tracing, and (depending on what you're up to) the AMDS and AvailMan tools can be used to locate processes holding blocking locks. (SDA, AMDS and AvailMan are obviously not APIs, so there's no good way to use these tools from a program. But they're very handy for poking around.)

In the longer-term, have a look at centralizing the core file I/O calls into (for instance) shareable image(s), of logging and tracing; of potential application enhancements. Features such as quiescing or cleanly exiting the open files when required; of coordinating and providing consistent backups of the data. Whatever you want to call the task; retrenchment, improvements, optimization, reducing support costs, integrated debugging, streamlining...

[Robert Brooks_1]

Richard wrote . . .

I think you'll find that it's only a nicety of the F$GETDVI lexical to convert it to a hexidecimal string on output and that sys$getdvi returns a lovely 16 byte dvi$_locknam as usual. And the volume lock living in bytes from 2 for 12.

RAB: Yes, you are correct; the SYS$ variant does work as expected. However, the LIB$ variant (in addition to the DCL lexical) also behaves in an undocumented (and wrong) fashion.

Richard also wrote . . .
But then I'm sure someone that works with VMS i/o would know that, so I'm going to write a little tester to make sure.

RAB: Touche!

-- Rob

[Richard J Maher]

Ok this is beginning to sound too difficult.

See below for a snippet that looks to come from a "Perfect Disk" patent application I found on the web.

So I'd be more than happy with the serialization locks F11B$s and the arbitration locks F11B$a but judging by my SDA (on a system that is not in a cluster) you don't get arbitration locks at all when you open a file. Have I stuffed up somewhere or, on single node boxes, are we forced to walk some FCB chain?

Someone got the Internals book handy?

(Is there a bit of deja-vu creeping in here? This is one of those posts that comes round every few years isn't it?)

Cheers Richard Maher

<<<<<<<<<<<<<<<<<<<<<<

When PD determines that it would like to move a particular file, it starts by taking out a â file serializationâ lock in â protected readâ (PR) mode with a â blocking ASTâ (the blocking AST causes a notification if another user tries to take out an incompatible lock). While it holds the F11B$s lock in PR mode, no other users in the cluster can change the state of its access. In particular, if no other user has the file open, then no other user can access the file while the lock is held.

After PD acquires the lock, it checks locally to determine if another user has the file open locally. This is done by searching the file control blocks (FCBs) maintained in main memory by the XQP for all open files. If it is not open on the local node, then PD takes out a â file access arbitrationâ lock (referred herein as the F11B$a lock) in null (NL) mode. If a file is open on any node in a VAXcluster, then there exists such a lock on that node. PD can then do a $GETLKI (get lock information) system call and determine how many such locks exist in the cluster. If there is more than one (PD's lock), then another user has the file open and PD will not attempt to move the file. PD then drops the F11B$a lock since it has no further use for it at that time. Assuming the process is to continue, PD then allocates space on the disk at the target location for the defragmented/optimized version of the file. It reads the file data from the old location and writes it to the new location. A verification pass can be performed if desired to guarantee that the data was correctly copied. Up to this point, if the system crashes for some reason, the old file exists as always and there is no problem. The space allocated for the new version of the file will be deallocated when the disk bitmap is rebuilt, a normal operation at start-up.

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

PS. Nice overview Hoff but a bit more detail on the resource-name and hierarchy convention for file-locks would be much appreciated.

[Richard J Maher]

Yep, here it is: -
http://groups.google.com/group/comp.os.vms/browse_frm/thread/46072b43d425a042/3b4112f70423e4b2?#3b4112f70423e4b2

Memory's not what it used to be.

Cheers Richard Maher