Strange result with processing search lists

[Jan van den Ende]

Our development team ran into some strange behavior.

They were able to boil it down to the handling of search lists.

I have appended a (pretty minimal) reproducer.

In the beginning of the procedure are 4 specifications of the same file (AUTHORIZE.EXE, but that is just a pick because that will exeist on any system).

Pick each one of those, to show the change in behavior.

Two of the specifications include (directly: SYS$SYSROOT; or indirectly: SYS$SYSTEM) a search list. Two others do not (directly, via SYS$SYSDEVICE, although we also tried its fysical name; or via SYS$COMMON).

The procedure processes the same file spec repeatedly.

And now for the surprise: using a search list specification, after some repetitions it will suddenly enter the main ELSE branch. No error or anything.

The number of repetitions is pretty constant on any node, but varies from about a dozen to about 70 on different nodes.
Strangely enough, on ONE node (which I personally have no account on) it is reported to have run 1000 loops without branching.
The really weird thing is, a filespec without search list reaches at least 10000 loops.

The phenomenon was detected on an application-defined search list, the system root example is just an generally testable reproducer showing it to exist there as well.

The routine shows a commented-out DUMMY search to try to reset the context. That makes no difference.

I DID try to add a one minute wait just a few iterations before the strange branch, to try and catch process statistics, but operational activities prevented me from trying them today. I hope tomorrow they will provide extra info, but in the mean time, I like
YOU
to have a look/show your insights/tell us why/share your view.

If it somehow has to do with some quota or stack running out, I consider the fact that that is happening _SILENTLY_, _WITHOUT_ any message, to be a bug; and it will be hard to convince me otherwise.

So,
may I invite your vision and/or experience?

Proost.

Have one on me.

jpe

[Jan van den Ende]

Next try on attachment:

Proost.

Have one on me.

jpe

[Willem Grooters]

He jan,

I didn't try it (too short term, will be off for a time after tonight) but one thing stroke me that might xplain the behaviour - somewhat.

The reproducer shows:

f$search("''wtm_file'.*",cnt) *so WITH context)

in one branche and

f$search("''wtm_file'") (without context)

in the other.

Since the contents in the first branch will change with every loop (since cbt will be increased) f$search will restart the search. The second branch will use the default context and so expire when the list is exhausted.

Try specifying a fixed contents on both, or none at all. What would the outcome be?

(I'll read the answer over two weeks)

Willem

[Hein van den Heuvel]

Patient: "Doctor it hurts when I do this"
Doctor: "Stop doing that, and you'll be fine"

What is the purpose of more than a few active SEARCH context in this, or any, application ???

Which application can really usefully keep track of that? DCL+RMS certaintly need lots of memory to remember each context, ready to continue where the application left off, but here the application never comes back.
DCL memeroy is limited, and may vary from system to system, and may be context (sorry!) dependend. That is... how many files does dcl have open, how many symbols,...

So my guess is that DCL/RMS simply runs out of memory and fails to nicely report that.
There is no 'status' for f$search so there is no good way to communicate a problem other then not retruning a name.

If the application insists on using fresh context identifiers all the time, then it will need to learn to 'clear' any old context which is no longer needed.
It can do so by repeating the f$search untill an empty string is returned.

Or it could re-use the context with a new search string, thus closing the old, starting a new.

The difference in reaction from the various ways of specifying the path, is the complexity of the context chosen. A simple wildcard filespec vs a searchlist.

Hope this helps,

met vriendelijke groetjes,
Hein.

[Jan van den Ende]

Thank you both.

So much for quickly trying a reproducer without thourough inspection.
I had not yet noted the new stream id for every search :-(

(the development team has accepted the explanation, and are addressing the issue)

Upon your note, I moved the "dummy" search into the first branch and uncommented it.
1000 loops without breaking.

That tickled my curiosity, so now I tried again without dummy search, because I still was intrigued by the difference in using/not using search lists.

I _WAS_ able to have it blow with non-searchlist filespecs, _BUT_, now it consistently blew with the expected SYMOVF overflow error.
And from the MUCH larger number of loops I deduce some understanding of just HOW demanding searchlist processing is (!!)

So, the issue now changes into:
WHY does a symbol overflow error escape getting noticed in searchlist processing?

To summarize once again:
unexplained different behavior in reacting to symbol buffer overflow in processing filespecs in searchlists compared to non-searchlist operations.

Anyone any reactions?

Proost.

Have one on me.

jpe

[Guenther Froehlin]

In my testing it was the F$PARSE which failed when it returned "" and not ".".

And it failed after only 7 search contexts. I would imagine the structures for the F$SEARCH and F$PARSE are kept in an area sized by PIOPAGES.

/Guenther