Are POSIX and native locks in any way compatible?

[Ben Armstrong]

Before I invest too much time in this, are POSIX and native locks in any way compatible? Could you lock a record in a file using native locks and have that lock be obeyed by a second process using POSIX locks, or vice versa?

Thanks,
Ben

[Hoff · Accepted solution]

Good question.

I would not stake my application run-time on it, barring finding documentation that says it works, or a clarification from HP support indicating it is permissible.

The underpinnings of the POSIX locks -- also known as byte-range locking -- are OpenVMS locks, though the way POSIX operates is quite different than how OpenVMS itself locks data -- Unix wants byte-range, and OpenVMS wants record-level.

I'd tend to treat this like mixing memory management calls -- you can use the various schemes within the same application, but not for the same objects. This pending any clarification of the documentation from the folks at HP support.

My reading of the V8.3 fcntl documentation (and my possibly faulty recollection of the implementation) does not indicate that you can nor should mix the two schemes.

[Hein van den Heuvel]

Posix byte range locks are simple USER MODE locks... with a byte range.

Note, they are a very recent addition to OpenVMS (8.3+ C RTL manual 2006)

Record locks are EXEC MODE resources, under an EXEC MODE file lock (volume+file-id). They effective lock just the first byte for a sequential file record.

Those locks are not compatible. Period.

Details:

Trivial test program

#include
#include
#include
#include
main (int argc, char *argv[])
{
int stat, file;
struct flock x = {F_RDLCK, SEEK_SET, 123, 456, 0};
file = open( argv[1], O_RDWR );
stat = fcntl( file, F_SETLK, &x);
return (close ( file ));

Running with debugger shows a lock:

C$__$9$DKB200932900021024 Granted at PR 0000007B-00000242

0x7B = 123
0x0242 - 0x7b = 455

The C$ makes it C-rtl owned
The DKB200 was the disk for the file.
The 932900021024 is the most bizarre representation for a file ID I have ever seen (and i have seen a few!)

The actual file ID was (271473,2,0)

So they took each 16-bit word and converted those to decimal creating a veriable length string ?!

Hope this helps someone someday.
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Willem Grooters]

There are two different things.

Record locks are set and maintained by RMS - native and EXEC by nature. The resourcename is derived by RMS in some fashion - but because the same code will be executed by ANY program accessing the record, it will be the same for each program. This causes ANY program accessing this record using RMS will have to obey the rules of RMS. If the resource (= record) is locked by one program, it will be locked for access to any other program. Theer is no way to get around this (except for hacking in the system).

POSIX locks - being native USER locks but with a specific way of resource naming - are set by the program. Unless there is a superivisor program taking care of the locks, each program will have to use the name the resource (file, record) in the same manner - be it a fxed name, or derived from the data (record key, for example) - and HOPE that other programs will use the same scheme and keep to it. If a program chooses NOT to, its free to do so.

Having said that, it will be clear (I hope) that you _can_ use both in programs, but NOT on the same resource!
For file access per-se, you may assume that RMS does it's job properly and tranparantly. Don't use PSIX locks here!

If you have your OWN resources - memory, for instance, or the file to access - you can use user-mode locks, but ALL programs using the same resource must use the same resource, naming scheme and access method.

My recommendation in that case would be to create routines taking care of any action and use these routines for access by all programs, to assure all programs use the same code (and therefore the same resource, naming scheme and access method). Usage of these routines should be mandantory - and AKAIK there is no means to ensure this other than convention and code standards.

Therefore, if your file access is done using RMS, I'd rely on RMS to do the locking. It simply works. Use user-mode locks only for specific, application-local resources, and use a library of routines to handle them.

[Brad McCusker]

I know this thread is closed, but, for future reference, I would like to add some emphasis.

Willem wrote:

>and HOPE that other programs will use the
>same scheme and keep to it. If a program
>chooses NOT to, its free to do so.

This is such an important point, and, a point that may seem foreign to long time VMS programmers. POSIX locks are advisory. From the POSIX spec:

"For advisory file record locking to be effective, all processes that have access to a file must cooperate and use the advisory mechanism before doing I/O on the file."

The OpenGroup spec (Issue 6) can be found here: http://www.opengroup.org/onlinepubs/009695399/toc.htm -

Good luck,

Brad McCusker (The C-RTL Project Lead when Byte Range Locking was being added)

Software Concepts International
www.sciinc.com

[Jan van den Ende]

Brad wrote:

>>>
POSIX locks are advisory.
"For advisory file record locking to be effective, all processes that have access to a file must cooperate and use the advisory mechanism before doing I/O on the file."
<<<

Tranlated in plain English, this means, that the locks only exist for those that care to look.

It is _NOT_ you that can lock, and so prevent access. If you lock, that only puts up a sign to those who are interested in potential signs, that they are advisded NOT to use the resource. _NOTHING_ prevents access by those who refuse or forget to look.

I am just surprised that they had the guts to call it "lock".
I for sure would not like such locks on _MY_ home!

fwiw

Proost.

Have one on me.

jpe

[Jon Pinkley]

The VMS Distribute Lock Manager is no different in that respect, but it does allow locks to be created in privileged modes, and thus, RMS can enforce any user of its services to wait until the lock is obtained.

But user mode applications using $ENQ and $DEQ must cooperate with each other, just like POSIX locks.

But as Jan says; it's like someone setting the SSID on his unsecured Wi-Fi access point to "PLEASE-STAY-OFF-THIS-NETWORK", which my sister in law actually saw while on a business trip.

[Hein van den Heuvel]

> The VMS Distribute Lock Manager is no different in that respect,

Right, you ask for the lock and get a result. If you choose not to aks, nothing will stop you. The lock in general does not have a magic key that opens access to the data, allthough the EOF pointter for rms sequential files comes close.

>> RMS can enforce any user of its services to wait until the lock is obtained.

Only for those actually using RMS.
A program can open an RMS managed file shared and read an write whatever, whenever it feels like. Only access through RMS record services is protected, but volontueerd. Now I grant you that it is a little tricky to understand indexed file record layout, but not impossible. Several programs provide non-interlocked read access (my Tunecheck, EGH's 'select', Fekko's DIX...) write access is equally possible, but of course not done.

fwiw,
Hein

[Willem Grooters]

Put it this way: The good thing of using RMS is that locking is mandantory, though hidden, simply by the use of RMS. If you bypass RMS and access the files directly, you will be able to ignore any possible locks and just go ahead - with all consequences.

[quote]
Several programs provide non-interlocked read access (my Tunecheck, EGH's 'select', Fekko's DIX...) write access is equally possible, but of course not done
[/quote]

...if, and only if, the developer has the dicipline to adhere to the scheme. And that is where the danger lurks.

[Ben Armstrong]

All very interesting. Yes, I at least understood the distinction about advisory locks in a basic way, but thanks for the elaboration on the topic.

Given that our application consists of cooperating processes that all follow the same rules for access because we have ensured that they do so through coding standards, we could make use of POSIX locks on the new platform. However, for VMS, I now understand we need to continue to use RMS locks so long as we continue to have old code that is not recompiled to take advantage of the new, platform-independent locks (and probably clear through to the end of life of the software on the VMS platform).

Thanks, everyone, for your time answering my questions.

[Robert Gezelter]

Ben,

While I have no idea of the structure of the code, in general it should be possible to hide the locking at a very low level routine.

As such, the use of POSIX vs. OpenVMS RMS locking would be transparent to the rest of the codebase. It is not particularly difficult to write the two alternative gatekeeper routines, and it significantly simplifies the codebase.

As you may imply from the above, I have used this approach many times for many purposes, not just for locking.

- Bob Gezelter, http://www.rlgsc.com

[Willem Grooters]

My impression is your code accesses files, on VMS using RMS (which includes locks) and directly on other platforms.
In that case, I would concentrate all file handling in one library, that is platform specific. By that,all detail involved is hidden. This includes the required locking - either implicit or explicit.
Your base code would just call these libary routines, regardless the platform - and therefor use the locking mechanisnms without even knowing; the technical detail will be outside the most important code: you business logic.

[Ben Armstrong]

Exactly. The code is already structured this way, with one routine mediating all access to the file, so it will not be hard to do.