topic Re: F$SEARCH failed in Operating System - OpenVMS

F$SEARCH failed

Wim Van den Wyngaert — Wed, 26 May 2004 09:30:53 GMT

I do a f$search on 1 file several thousands of times a day. With success because the file always exists.

One day, f$search returned an empty string and this while the file did exist !!! Only once !

I found out that the only thing running was a procedure that was setting the protection of the file and the directory it is in.

I wrote a procedure to try to simulate this situation but f$search never failed.

Suggestions of what went wrong ?
I have no sys$ouput or sys$error and have never had this before during the 3 years that is is executing.

Re: F$SEARCH failed

Uwe Zessin — Wed, 26 May 2004 09:57:03 GMT

Sounds like a typical race condition - you can try it a million times now, but it will never happen again.

The update could have locked the file header, but I think F$SEARCH() should have caused an error condition and not silently return an empty string.

On the other hand it is possible that the DCL symbol was already pre-loaded with an empty string, and F$SEARCH() did return a warning, but it was silently ignored. If I recall correctly, F$SEARCH() does not update the symbol when it encounters an error.

You could load the symbol with an 'impossible' filespecification and check the contents after the call to F$SEARCH().

Re: F$SEARCH failed

Wim Van den Wyngaert — Wed, 26 May 2004 10:05:47 GMT

Uwe,

A "on warning" was active. So, no error or warning was given. The symbol was not initialized to "" by the procedure.

I tried to do a set prot in a loop and parallel the f$search, also in a loop. After 10 minutes there were still no problems.

Re: F$SEARCH failed

Antoniov. — Wed, 26 May 2004 11:33:34 GMT

Wim,
if you have error and warning trap and you haven't received any error, the only reason of F$SEARCH() empty was the file didn't exist!
I guess your file may be deleted and rewritten or else may be procedure write always new version (as edit) with some purge command; in this situation, you could read file in very little time when file didn't exist.
As Uwe dixit it a race condition!

Antonio Vigliotti

Re: F$SEARCH failed

Petr Spisek — Thu, 27 May 2004 01:39:05 GMT

F$Search() returns "" when wildcards in filename is using and condition is completes (list of suiting files). Do you use any wildcards in this case?

Re: F$SEARCH failed

Wim Van den Wyngaert — Thu, 27 May 2004 01:39:22 GMT

I was able to simulate it. It does seems to be a feature/bug.

>ty wim1.lis
$a:
$ set file wim.dir/prot=w:rwed/own=system
$ set file [.wim]*.* /prot=w:rwed/own=system
$ goto a

>ty wim2.lis
$ set ver
$ on warning then exit
$ a:
$ if f$sea("[.wim]wim.lis") .eqs. "" then exit
$ goto a

I submitted the 2 files in batch on a system not doing anything else.

>ty wim2.log
08:32:29.83 $ set ver
08:32:29.83 $ on warning then exit
08:32:29.83 $ a:
08:32:29.83 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.86 $ goto a
08:32:29.86 $ a:
08:32:29.86 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.86 $ goto a
08:32:29.86 $ a:
08:32:29.86 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.86 $ goto a
08:32:29.86 $ a:
08:32:29.86 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.87 $ goto a
08:32:29.87 $ a:
08:32:29.87 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.87 $ goto a
08:32:29.87 $ a:
08:32:29.87 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.88 $ goto a
08:32:29.88 $ a:
08:32:29.88 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.88 $ goto a
08:32:29.88 $ a:
08:32:29.88 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.89 $ goto a
08:32:29.89 $ a:
08:32:29.89 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.89 $ goto a
08:32:29.89 $ a:
08:32:29.89 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.89 $ goto a
08:32:29.89 $ a:
08:32:29.89 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.91 $ goto a
08:32:29.91 $ a:
08:32:29.91 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.91 $ goto a
08:32:29.91 $ a:
08:32:29.91 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.91 $ goto a
08:32:29.91 $ a:
08:32:29.91 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:29.91 $ goto a
08:32:29.91 $ a:
....
08:32:30.40 $ a:
08:32:30.40 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
08:32:30.40 $ goto a
08:32:30.40 $ a:
08:32:30.40 $ if f$sea("[.wim]wim.lis") .eqs. "" then exit
SYSMGR_WVW job terminated at 27-MAY-2004 08:32:30.43

Re: F$SEARCH failed

Wim Van den Wyngaert — Thu, 27 May 2004 02:41:07 GMT

Further test results.

1) VCC_FLAGS=0 -> same problem
2) 6.2 1H3 -> same problem

Re: F$SEARCH failed

Jan van den Ende — Thu, 27 May 2004 04:37:50 GMT

Wim,

>ty wim2.lis
$ set ver
$ on warning then exit
$ a:
$ if f$sea("[.wim]wim.lis") .eqs. "" then exit
$ goto a

2 conditions, both EXIT procedure

Q&D: change to $ EXIT '$STATUS

other proposal:

$ On warning then goto _warning
$ if f$sea .eqs. "" then goto _EMPTY

$_warning:
---- MAIL or WRITE SYSOUTPUT or .. your favorite signalling for " At warning -- $STATUS= "
$ goto a ! Is this fleeting or persistent?

_empty:
--- similar

This way you DO make the distinction between some condition on one hand, or just FNF on the other!

From WIM1.LIS it is clear that the file definitely STAYS in existence. This rules out Antonio's suggestion (although the idea in itself was plausible enough to require verification!)

You DO have me curious!!

Re: F$SEARCH failed

Wim Van den Wyngaert — Thu, 27 May 2004 04:45:38 GMT

Jan,

Tested it. It is not the warning condition but f$search that fails.
You can test it yourself on your machines. It should be the same.

My guess is that f$search considers "file or directory locked" as "file doesn't exist". While it should wait or return an error (or better : it should be an option to f$search).

Re: F$SEARCH failed

Wim Van den Wyngaert — Thu, 27 May 2004 05:00:40 GMT

FYI the status is %X00010001 when the f$search fails.

Is it possible that an warning status is given without a message ? Never seen that !

Re: F$SEARCH failed

Ian Miller. — Thu, 27 May 2004 05:07:53 GMT

I suspect its an unreported error possibly a lock conflict. Parhaps check $status after the search just in case?

Re: F$SEARCH failed

Eberhard Wacker — Thu, 27 May 2004 05:19:14 GMT

Hi Wim,

when the directory file is accessed at the time of your prot/owner setting then all files in the directory are locked.
f$search is not able to report this (for me it's a bug, but maybe engineeering has another opinion reagrding this), it returns the file name or a null string.
If you replace the f$search with e.g. f$file_attributes("wim.lis","locked") (and let only the directory file modification line in yout test procedure) then you will see this very fast.

Cheers
Eberhard

Re: F$SEARCH failed

Wim Van den Wyngaert — Thu, 27 May 2004 05:25:13 GMT

Eberhard,

I fully agree : this is a bug.
It makes almost all procedures unreliable.

Wim

Re: F$SEARCH failed

Eberhard Wacker — Thu, 27 May 2004 06:36:08 GMT

Wim,

correction of what I've written: not the files itselves are locked, the
directory file and so the records of the files therein are locked.

Execute your reduced procedure:
$a:
$ set file wim.dir/prot=w:rwed/own=system
$ goto a

and try to save a file in the directory (via copy or editor):
often this will have no success.

E.g.
$ cop test.txt [.wim]
%COPY-S-COPIED, USR2:[TEST]TEST.TXT;1 copied to USR2:[TEST.WIM]TEST.TXT;1 (1 blo
ck)
$ cop test.txt [.wim]
%COPY-S-COPIED, USR2:[TEST]TEST.TXT;1 copied to USR2:[TEST.WIM]TEST.TXT;2 (1 blo
ck)
$ cop test.txt [.wim]
%COPY-E-OPENOUT, error opening USR2:[TEST.WIM]TEST.TXT;1 as output
-RMS-E-FLK, file currently locked by another user
%COPY-W-NOTCOPIED, USR2:[TEST]TEST.TXT;1 not copied
$ cop test.txt [.wim]
%COPY-S-COPIED, USR2:[TEST]TEST.TXT;1 copied to USR2:[TEST.WIM]TEST.TXT;3 (1 blo
ck)
$ cop test.txt [.wim]
%COPY-E-OPENOUT, error opening USR2:[TEST.WIM]TEST.TXT;1 as output
-RMS-E-FLK, file currently locked by another user
%COPY-W-NOTCOPIED, USR2:[TEST]TEST.TXT;1 not copied

and

$ delete/noconf/log [.wim]test.txt;*
%DELETE-W-SEARCHFAIL, error searching for USR2:[TEST.WIM]TEST.TXT;*
-RMS-E-FLK, file currently locked by another user

All depends how fast and often you are accessing (better said: MODIFYING) the
directory file. I'm now quite sure that this is the so called intended behaviour.

Eberhard

Re: F$SEARCH failed

Wim Van den Wyngaert — Thu, 27 May 2004 07:09:54 GMT

Are there any other means of locking a directory ?

Re: F$SEARCH failed

Jan van den Ende — Thu, 27 May 2004 08:04:40 GMT

And, much more important/useful:

Is there a way to TEST for this lock, and specify HOW to react upon the lock (eg, as now = fail-but-do-not-report, but also fail-but-report, or wait-forever-until-unlock, or wait-till-timeout-or-unlock, or .. anyone with other options?

As this now is NOT possible (to the best of my knowledge), then, (and I know you read this), please, Guy Peleg, can we put this on your list?

Jan

Re: F$SEARCH failed

Eberhard Wacker — Thu, 27 May 2004 11:27:06 GMT

Sample for a (not very fine) workaround:

$ set mess/nofac/nosev/noid/notext
$ a:
$ if f$search("[.wim]wim.lis") .eqs. "" ! maybe not there
$ then
$ chk = f$file_attributes("[.wim]wim.lis","rvn")
$ if $status .eq. %X00000910 then exit ! definitely not there
$ endif
$ goto a

Re: F$SEARCH failed

Wim Van den Wyngaert — Fri, 28 May 2004 01:14:22 GMT

Eberhard,

Only, when the file is exclusively open, the f$file fails. So, not yet perfect.

Wim

Re: F$SEARCH failed

Jan van den Ende — Fri, 28 May 2004 01:48:00 GMT

Wim, Eberhard,

a second, and more serious, flaw in this scheme is the timing.
If locked during f$search, the lock may well be gone by the time f$file is executed!

Jan

Re: F$SEARCH failed

cbkteam1 — Fri, 28 May 2004 02:39:19 GMT

Wim, Jan,

I see no timing problem, the status code is that from "%SYSTEM-W-NOSUCHFILE, no such file". This is only returned, if the file is definitely not there. In case of lock (of the directory or the file itself) the loop goes further on. In principle in this special case you could discard the f$search completely.

Eberhard