Deleting a File that is Open

[Hoff]

In addition to Bob G's materials referenced earlier, there's http://labs.hoffmanlabs.com/node/617 that might get you going on ASTs, and there are links off to processor cache coherency and barriers and interlocked instructions and synchronization, and off to various code examples from there.

The usual distributed coordination and interprocess notification technique here would involve locks the lock manager and http://labs.hoffmanlabs.com/node/492 and such.

A distributed I/O mechanism (whether in the form of QIOserver or one of the more widely available grid engines such as MPI or BOINC or Xgrid) or access to features including checkpoint-restart would and related features be useful here, as would have been an API for synchronizing applications and caches with BACKUP activities or with a shadowset split DISMOUNT. That written, these mechanisms are not available with OpenVMS. RMS and the lock manager are available, however, and the application itself can be coded to take the necessary steps to ensure a reliable and consistent archive of the application data. If you're interested in adding transactional control and the ability to run a journal, there are both integrated (though separately licensed) options and there are add-on options and application protocols (eg: paxos) that can be used.

[John Gillings]

Howard,
Just winding back... I'm trying to understand what you're attempting to achieve.

You have process 1 with an open channel to a file.

Process 2 deletes the file, and you're expecting process 1 to magically look at a new file with the same name? Won't happen!

Answering your direct questions...

Looking at the deleted file. Yes, the directory entry has been removed and the file set erase on delete. Yes you can dump the header of the file, using the FID. Get the FID from SDA:

Channel CCB Window Status Device/file accessed
------- --- ------ ------ --------------------
0010 7FF06000 00000000 DSA1:
0020 7FF06020 88B03C40 DSA1:(2745,8382,0)

So, the index number is 2745. Simplest way to find the header is with DUMP/FILE. You need an offset into INDEXF.SYS. I thought there was a /INDEX qualifier? Use the FID as a first guess then find the offset as the error:

$ DUMP/FILE DSA1:[000000]INDEXF.SYS /BLOCK=(COUNT:1,START:2745)

look for resulting FID:
Virtual block number 2745 (00000AB9), 512 (0200) bytes File identification: (2183,5,0)

$ WRITE SYS$OUTPUT 2*2745-2183
3307

This is the block number of the header in INDEXF.SYS:

$DUMP/FILE DSA1:[000000]INDEXF.SYS /BLOCK=(COUNT:1,START:3307)
...
File identification: (2745,8382,0)
...
File characteristics: Marked for delete
...
Identification area
File name: DELETEME.TXT;1

You can work out the directrory the file was in by following the backlink.

DFU can do a lot of the hunting for you.

Question 2...

Terminating the image is simple. Use SHOW DEVICE/FILE to list the processes with open files. Use STOP/IMAGE to terminate the image, or STOP/ID to terminate the process. Simple matter to write a procedure to do it automatically, BUT you need to wait until the images run down and the files are closed before dismounting the disk. Killing just the image may also lead to other images executing, opening other files, etc...

Also, beware process permanent files. These will not be closed just by terminating the image. You'll need to take out the process to close them.

[Hein van den Heuvel]

Jon Pinkley wrote: "What do you expect to happen to the application that has the file open if you somehow force a dismount? It isn't going to be able continue without error."

IMHO this is the most important observation. You might as well shoot the process. Only if you know the file will never ever be used again, could you possibly get away with an outside tool, to put some code in the pool to close a file (dummy fab with BLN,COD and IFI filled in) , and queue and AST to the process to run that in its context. This _hinted_ to in http://www.openvms.compaq.com/doc/82final/5841/5841pro_013.html
" Code that must perform other operations in another process's context (for instance, to execute a system service to raise a target process's quotas) can be written as an OpenVMS Alpha or OpenVMS I64 executive image, as described in Section 4.7.2 "

As Howard himself indicated and many echoed... Just close the file after the operation, with the performance overhead of that (if you need to re-open the same file), and the software revision price to pay.

Async, on request, file close/reopen may be cute but overkill.


John Gillings wrote:

"I thought there was a /INDEX qualifier?"

There is. /IDENTIFIER

But the implemenation is broken. (IMHO!)
It uses F$FID_TO_NAME, which is cute but not asked for, and then refuses to make do if that effort fails. F$FID_TO_NAME visits then directory in vain.

So in the marked-for-delete case you get:

$ dump/head/id=72197 sys$login
%DUMP-E-READFIDHEADER, error reading file header for file ID (72197,179,0)
-SYSTEM-W-NOSUCHFILE, no such file

Or when a file is locked exclusively you'll get:

$ dump/head/id=72197 sys$login
%DUMP-E-OPENIN, error opening _EISNER$DRA3:[DECUSERVE_USER.HEIN]TMP.TMP;1 as input
-RMS-E-FLK, file currently locked by another user

The method for DUMP/FILE_HEADER from INDEXF work in both case. Below you'll find a little command file to facilitate the math.
Usage example:
$ @DUMP_FILE_HEADER_BY_ID.COM sys$login: 72197
:
File characteristics: Marked for delete
:

hth,
Hein.

$ type DUMP_FILE_HEADER_BY_ID.COM
$!
$! read_file_header.com Hein van den Heuvel, July 2006
$!
$! This command file (Hack!) dumps the header of file,
$! even if it is locked or deleted... if you have the file ID (SDA> SHOW PROC/CHAN )
$!
$!
$! 2) Use file Id to look up file corresponding file header in INDEXF.SYS
$!
$if p1.eqs.""
$then
$ write sys$error "Please provide a DEVICE and FILE-ID as arguments"
$ exit
$endif
$dev = f$parse(p1,,,"device")
$id = p2
$!
$! The file-ID is an offest into INDEXF.SYS
$! INDEX.SYS first starts out with 4 cluster, and then
$! a bitmap to hold a bit for each potential file header.
$! And as for every VBN, the count starts at # 1.
$!
$indexf_bitmap_vbn = (f$getdvi(dev,"cluster") * 4) + 1
$ibmapsize = (f$getdvi(dev,"maxfiles") + 4095) / 4096
$header_vbn = indexf_bitmap_vbn + ibmapsize + id -1
$dump/file_header 'dev'[000000]indexf.sys /blo=(start='header_vbn',count=1)

[HDS]

Wow.

Such a wealth of information. Thank you all.

As a whole, you have been able to provide supporting information to my position that I am going to have to change the application. I figured that it was worth asking if there was some sort of "trick" that I could use here without having to modify the application, but...looks like there is not. That is okay.

To respond to some very valid points:
- If that file could be deleted, the application [as I have found] does not go back and attempt to re-read from that lun...so there is no attempt to "read that newly replaced file." I agree, that could never work. I had to tear down the application code (this section of the module is deep within well-nested subroutines) and have since found that there are no attempts to re-read from that lun without first checking to verify that the lun is open. So, the good thing (for me) is that the pointer does not need to be maintained. This very much simplified my getting "approvals" to add and test the simple adding of that single CLOSE statement. We are all a tad surprised that it was not there already. The performance aspect aside (this is not something that happens very often and, when it does, it is just once), we here internally believe also that it is the best approach.

- We (as a development department) are working more and more with AST synch methods...I agree...that would have been nice to have already in the code here. We are working towards more advanced and more flexible synch methods. The links to the presentations will be very useful to me and, likely, several others. Thank you for them. Unfortuantely, adding that technique here will require more mods to legacy code than that for which I am permitted to perform, so such could not be done here in this case.

I truly thank all of you you for all of this information. For me, this forum continues to be like "the wrench to the plumber."

-H-

[HDS]

Hello.

I am grateful for all of the responses. In spite of the fact that no magic trick solution exists for me on this one, the responses were very informative and supported my initial instinctive hunch.

Thank you all.

-H-