topic Re: Reading locked files in Operating System - OpenVMS

Reading locked files

mpradhan — Thu, 17 Mar 2016 22:10:18 GMT

Hello,

I have locked files, I am looking for ways to read them via sys$open() and sys$read() using block IO. Is there any FAB flag that I must use to let this happen?

I have a test code which keeps the file locked using fcntl, read shared or exclusive. Now, if I try to open the same file from another application it fails with error '%RMS-E-FLK, file currently locked by another user'.

Any help is appreciated.

Thanks,

Manoj

Re: Reading locked files

Hein van den Heuvel — Thu, 17 Mar 2016 23:38:56 GMT

Do you have control over the code/program for the second user?

The first accessor may/will define what other can do through the FAB$M_SHR.

To allow other reader, specify FAB$V_SHRGET, to allow writers FAB$V_SHRUPD ( and/or PUT, DEL ... all the same really).

Now any subsequent opener must also approve of prior opens.

So if yo specified a WRITE option in FAB$M_FAC, then the other SHR has to allow write otherwise they'll get that FLK error.

For BLOCKIO (readers0 I suspect you want look at using FAB$V_UPI

Finally if you want to really blow through exclusive access, like BACK/IGNORE=INTERLOCK, then check out the IO Reference manual and look for FIB$V_NOLOCK

hth,

Hein

Re: Reading locked files

mpradhan — Fri, 18 Mar 2016 14:57:11 GMT

No, I dont have access to the other program which might have locked the file, this could be an applicaiton writing to log files, etc.

I have tried Block IO with V_UPI to no avail.

I cannot use BACKUP /IGNORE=INTERLOCK.

All my code is sys$open based and hence moving to sys$qiow would be quite a change to just acheive a read access to a locked file :(.

This part of my example code which is trying to open.

pFab->fab$b_fac = FAB$M_BIO | FAB$M_GET; // block IO
pFab->fab$l_xab = (char*)&xabsum;

status = sys$open(pFab);
if (!(status & 1)) {
cerr << "sys$open(" << _sFileName << ") failed error=" << status << endl; <--- I Get the error here when other program has the file locked (see below for other program code).
return -1;
}

Program that's locking the file (this is an example code)

h = open( argv[1],O_RDWR, 0666, "shr=nql" );
if( h < 0 )
{
perror("failed open a file");
return 1;
}
printf( "opened..." );

lk.l_type =RDLCK;
lk.l_whence = SEEK_SET;
lk.l_start = 0;
lk.l_len = 0;

rc = fcntl( h, F_SETLK, &lk );
if( rc != 0 )
{
printf( "\n" );
perror( "fcntl failed" );
close(h);
return 1;
}

printf( "file is locked.\n\nPress Enter to unlock..." );

Re: Reading locked files

Hein van den Heuvel — Fri, 18 Mar 2016 15:28:01 GMT

As I write in the first reply, if the first opener does not grant shared access then you are dead in the water.

The example shown does NOT grant access, for that to happen you need to add something like "shr=nql,get"

Please note that NQL is a minor performance option and does NOT change the (FILE) locking behaviour at all. It only applies for RMS RECORD locks, and the C-RTL is not using record mode untill you tell it to (CTX=REC).

Suggestion: to check out what is happening use ANALYZE/SYSTEM ... SET PROC "proc with program running" ... SHOW PROC /RMS=(RAB,FAB)

I added a SLEEP to help with that.

Hein

#include  <unistd.h>
#include  <stdio.h>
#include  <sys/types.h>
#include  <unistd.h>
#include  <fcntl.h>
main(int argc, char **argv) {
  int h, rc;
  struct flock lk;
  h = open( argv[1],O_RDWR, 0666, "shr=nql,get" );
  if( h < 0 ) {
    perror("failed open a file");
    return 1;
  }
  printf( "opened..." );
  sleep(1000);

  lk.l_type =F_RDLCK;
  lk.l_whence = SEEK_SET;
  lk.l_start = 0;
  lk.l_len = 0;
 rc = fcntl( h, F_SETLK, &lk );
 if( rc != 0 ) {
    printf( "\n" );
    perror( "fcntl failed" );
   close(h);
   return 1;
  }
  printf( "file is locked.\n\nPress Enter to unlock..." );
}

Re: Reading locked files

mpradhan — Fri, 18 Mar 2016 18:25:23 GMT

"if the first opener does not grant shared access then you are dead in the water"- holds true only if we are going sys$open() route, not if we are going sys$qio() route, correct?

since I have no control over the other process, is sys$qio() my only programatic opton to be able to read locked files?

Re: Reading locked files

Hoff — Fri, 18 Mar 2016 20:43:16 GMT

What's your goal? Those locks are there to ensure data consistency and integrity, after all.

Override those and the "eavesdropping" application can get corrupt data.

This is basically the same as yanking out a copy of the data from underneath a running relational database.

As for tools that do this, callable convert and callable backup are both available for generating the requested "data". Yes, both CONVERT and BACKUP are also available as callable APIs.

Or use the C extensions to RMS, as was mentioned earlier.

As with most folks that have written their own "backup" tools and those that have experimented with such "fun" as the imfamous BACKUP /IGNORE=INTERLOCK, successful results of these sequences are far from certain. Data can be cached, after all.

There are examples of more properly coordinating shared access around, too.

Re: Reading locked files

mpradhan — Fri, 18 Mar 2016 21:49:02 GMT

my goal is to be able to read a locked file, no matter what sharing option the application (the one which currently has the lock) had while opening the file for reading. I am aware that this will be a blind read and will result in inconsistent data.

arent the examples (under hoffman labs) and fab$b_shr for the application which opened the file for writing, i.e. the writer (or the first application which opened the file) controls how the file is shared with subsequent applications?

I have experimented with fab$b_shr and no matter what I do, if the locking applicaiton already has a lock and has no sharing with others, then my read application always fails to open the file.

If a swithc the code in reader application to use sys$qio instead, then I can open the file with V_NOLOCK.

Re: Reading locked files

Hoff — Fri, 18 Mar 2016 23:44:31 GMT

The share SHR settings are how the first application allows access to subsequent processes; how sharing is declared, and how the application tells RMS what can and should be cached in memory and what should be maintained on disk; how access should be coordinated. That pointer to that example was an attempt to show you how to add options to the fopen call, given your stated requirement that RMS calls cannot be (directly) used.

Since you want inconsistent data, the switches you need here are RAB$V_NLK and RAB$V_RRL. Those disable locking and enable what's known as read regardless; you get whatever happens to be on disk at the time, though you should not expect to reliably receive errors or diagnostics even in the event of a locking-related collision.

Now to specify these settings on a fopen? See the code example mentioned previously and see the help on the creat() function, and see the OpenVMS C documentation for creat() and fopen(), and see the RMS documentation for details on the switches. If you're on a not-ancient OpenVMS version, use the following command to get an overview of the options available on creat() , and which include the NLK and RRL settings:

$ help crtl creat arguments

Between that and the C documentation, the names of the NLK and the RRL switches (and the assocated RMS documentation for RAB$V_NLK and RAB$V_RRL), and the previously-cited examples of OpenVMS C fopen calls, you should be able to access the requested potentially-corrupt data.

Or if you're inclined to write somewhat less of your own code, just use callable convert (conv$convert) or callable BACKUP.

Re: Reading locked files

Hein van den Heuvel — Sat, 19 Mar 2016 04:32:21 GMT

>>> the switches you need here are RAB$V_NLK and RAB$V_RRL.

I call BS-squared.

- RAB options only play a role when you can get into the file. If the first opener choses to disallow that, like Hoff indicates himself, then that's irrelevant.

- NLK + RRL is inferior to, or with the right SET RMS equal to NQL, which is already requested. A seemingly reasonable but actually futile request.

>> Those disable locking and enable what's known as read regardless;.

Yes and no. Mostly no. NLK and RRL both still request a record lock, so locking is no disabled. But for NLK the lock is converted to NL immediately upon getting it, and for RRL the record data is returned with a warning it was not locked upon NOT getting the lock.

Now with SET RMS [/SYSTEM] /QUERY_LOCK=DISABLE the COMBINATION will indeed diable RECORD locking completely, but file locking stays in effect.

As Manoj indicates, only QIO ACCESS + FIB$V_NOLOCK will blow through that if the process r image has.SYSPRV.

Manoj>> If a swithc the code in reader application to use sys$qio instead, then I can open the file with V_NOLOCK.

Correct.

Hein.

Re: Reading locked files

abrsvc — Sat, 19 Mar 2016 10:53:40 GMT

As Manoj indicates, only QIO ACCESS + FIB$V_NOLOCK will blow through that if the process r image has.SYSPRV.

Manoj>> If a swithc the code in reader application to use sys$qio instead, then I can open the file with V_NOLOCK.

Correct.

Hein.

==================

Be warned (as stated before). This needs to be emphassized here.

Using this method will not cause a flush of the cache and may result in reading stale +/or incomplete data.

Dan

Re: Reading locked files

mpradhan — Mon, 21 Mar 2016 16:31:41 GMT

Guys - thanks, it's been quite a learning.

Now I feel, QIO is the routine, I would have to turn to, to be able to read (blind read) locked files, irrespecitve how the first application wished to share the file. This puts a great level of stress on my existing code base, which was using sys$open, sys$read etc so far.

Is there any documentation/example you could point me to, to be able to read attributes via QIO and map them to XAB blocks (XABSUM, XABALL, XABFHC, FABITM, XABPRO)? I see IO$_ACCESS can fill a struct atrdef, but mapping those to XAB blocks seem very daunting. Looking for some guidelines here.

Thanks

Manoj

Re: Reading locked files

mpradhan — Tue, 22 Mar 2016 20:46:26 GMT

Guys,

Do you know which library I'll be linking to if I use backup$start?

$ CXXLINK/exe=test_lock.exe test_lock.obj

%ILINK-W-NUDFSYMS, 1 undefined symbol:

%ILINK-I-UDFSYM, unsigned int backup$start(_buf_arg *)

%ILINK-W-USEUNDEF, undefined symbol unsigned int backup$start(_buf_arg *) referenced

source code name: "backup$start(_buf_arg*)"

section: .text

offset: %X0000000000001630 slot: 2

module: TEST_LOCK

Re: Reading locked files

Hein van den Heuvel — Wed, 23 Mar 2016 11:17:54 GMT

BACKUPSHR as to be expected.

I Checked using brute-force:

$ search sys$library:*.exe/win=0/log backup$start
:

%SEARCH-S-MATCHED, SYS$COMMON:[SYSLIB]BACKUPSHR.EXE;1 - 1338 records, 1 match

(Searches like that can fail if the string happens to fall over a 512 byte record boundary)

Hein.

Re: Reading locked files

mpradhan — Tue, 05 Apr 2016 16:55:54 GMT

what's the correct way of linking this using CXXLINK?

The example I saw in http://h71000.www7.hp.com/doc/84final/4493/4493pro_002.html says "this is a VAX C program", does it still work on IA64?

Re: Reading locked files

H.Becker — Tue, 05 Apr 2016 22:05:21 GMT

> what's the correct way of linking this using CXXLINK?

Hmm, I'm not sure I have the right context: what is "this"? On the other hand, on I64 you don't need CXXLINK at all. CXXLINK (CXX aka C++) is needed for linking C++ programs on Alpha (and maybe on VAX, I don't know for sure). CXXLINK is availbale on I64 just for convenience, for example if you use common command procedures/makefiles on Alpha and I64. On I64, just use LINK for C++ programs as well as other programs.

> The example I saw in http://h71000.www7.hp.com/doc/84final/4493/4493pro_002.html says "this is a VAX C program", does it still work on IA64?

I don't see a VAX C program on that page, I do see one on http://h71000.www7.hp.com/doc/84final/4493/4493pro_001.html#bottom_001. I didn't try that. VAX C is an old C implementation. Your C compiler may still accept /STANDARD=VAXC. However, this is not recommended. Try the default for /STANDARD, which is a relaxed C89, and fix all the errors/warnings. No matter how you compile, you link with the LINK command.

Edit: On Alpha it goes like "this" (and it shouldn't be different on I64):

$ cre bck.c
/* Example 3-1 Calling the Backup API with a VAX C Program */
...
^Z
$ cc/standard=relaxed bck
$ cre bck.opt
sys$share:backupshr/share
^Z
$ link bck,bck/opt
$

Re: Reading locked files

mpradhan — Wed, 06 Apr 2016 20:10:29 GMT

thanks. That opt file was what I was missing and I was hoping the same (content of opt file) could be taken at command line.

Thanks

Re: Reading locked files

H.Becker — Wed, 06 Apr 2016 23:29:04 GMT

On OpenVMS/Alpha V8.3, from HELP LINK:

Format

LINK file-spec [,...]

and from HELP LINK PARAMETER:

file-spec [,...]

   Specifies one or more input files (wildcard characters are not
   allowed). Input files may be object modules, shareable images,
   libraries to be searched for external references or from which
   specific modules are to be included, or options files to be read
   by the linker. ...

This is wrong. On the command line there can only be file-specs for object modules, libraries or options. That is, you can have the usual file types .obj, .olb and .opt. A shareable image file (.exe) is a linker input file, but you can not specify it as input on the command line.

However, there can be libraries with object modules and libraries with shareable images. So you can create your own shareable image library containing sys$share:backupshr.exe - essentially it contains the exported symbols of that
shareable image.

$ libr/cre/share bck.olb
$ libr/share bck.olb sys$share:backupshr

Then your link command is

$ link bck,bck/lib

And yes, you can delete the bck.olb after linking your main image: it's not needed to run your program.

In an options file you can specify a shareable image as input file. If it comes to specify own or non-system default shareable images (others than in sys$share:imagelib.olb), almost all VMS developers prefer using options files over shareable image libraries.

Re: Reading locked files

Tom Wade_1 — Sat, 16 Apr 2016 12:40:31 GMT

Don't know if this is too late, but check out RALF (Read a Locked File) at www.tomwade.eu/software