topic Re: File locking exhausts PGFLQUOTA in Operating System - OpenVMS

File locking exhausts PGFLQUOTA

Stehlik Tomas — Mon, 13 Jul 2009 09:28:31 GMT

Hi all,

Attached test program just locks 1 byte in file for read then escalate this lock for read-write than again lower it from exclusive to shared access and finally unlock it. If it runs in loop it allocate memory pretty fast increasing VIRTUALPAGECNT until exhaust PGFLQUOTA.

Documentation for fcntl says:
â All locks associated with a file for a given process are removed when a file descriptor for that file is closed by that processâ

http://www.openvms.compaq.com/doc/83final/5763/5763pro_033.html#index_x_888

It happens on IA64 OpenVMS with last patches.
Is problem in this test or system?

Re: File locking exhausts PGFLQUOTA

H.Becker — Mon, 13 Jul 2009 12:00:30 GMT

Your test looks fine to me. The documentation only talks about the locks being removed. It doesn't say anything about memory being used.. In that sense the documenation seems OK. But it looks like a sub-optimal behavior, which is either a bug or seems worth to be documented as a (current/temporary/???) limitiation.

Re: File locking exhausts PGFLQUOTA

John Gillings — Mon, 13 Jul 2009 21:27:15 GMT

This needs more investigation before calling it a bug. Memory depletion can be a leak, or it can be due to usage pattern pathology, like checkerboarding.

I'm not familiar with the C++ locking interface, so I don't know the expected lifecycle of the structures involved - perhaps there's some kind of "Free" operation? What happens if you remove the F_UNLCK? Can you keep bouncing between F_RDLCK and F_WRLCK forever? Is there a null lock mode? (ie the lock still exists, but it doesn't block any other lock mode).

Try adding calls to LIB$STAT_VM to see if there's a pattern of depletion. Try sampling all statistics at the bottom of each loop. Output a timestamp, and a comma separated list of statistics, including loop counters for both loops (i and j). Give it a T4 header, and you can examine the timeline of VM usage.

If you want finer detail, use FAKE_RTL to trace all calls into VM routines. This will tell you if there's a genuine leak, or some VM usage pathology.

Re: File locking exhausts PGFLQUOTA

H.Becker — Tue, 14 Jul 2009 08:50:36 GMT

>>>
This needs more investigation before calling it a bug.
<<<

True. Given the simple source and the fact that it does work as expected on other systems, for example Linux, the investigation should be done by HP.

The byte range locking (aka file locking) is defined in the X/Open standard. It is (more or less) new to the VMS CRTL. The source code (although it has the type cpp) is plain C. There are no user-allocated/-controlled data structures on the heap involved.

Re: File locking exhausts PGFLQUOTA

Stehlik Tomas — Tue, 14 Jul 2009 09:32:38 GMT

Heap Analyzer shows that changing lock level on the same offset and range allocates another memory SYS$EXPREG. Why it is not just reused?

In the mean time I found workaround to use in pairs for each lock also unlock not just to change lock level.

I start to investigate this while using Sqlite ported to OpenVMS where they for UNIX code branch that is used also for VMS just escalate and again lover locking level.

Re: File locking exhausts PGFLQUOTA

JagadishM — Tue, 14 Jul 2009 11:14:53 GMT

Hello,

This looks like an OpenVMS CRTL issue, and requires future investigation.

Please open a QUIX case with HP, with decc$shr.exe image information.

Thank you,
Jagadish M.