DCL anomaly

[Willem Grooters]

VMS 7.3-2 - and possibly others (didn't check yet).

If you use a non-existing symbol in a lexical function, DCL will signal an error, but when the same is used in creating a string, it's accepted as an empty string.

DKWS38>sho sym bagger
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
DKWS38>say f$fao ("!AS",bagger)
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\BAGGER\
DKWS38>say "dit is ''bagger'"
dit is
DKWS38>s="Dit is ''bagger'"
DKWS38>sho sym s
S = "Dit is "
DKWS38>

Sometimes it would be nice if this situation were signalled "Undefined symbol" as well - a typo would less likely be overlooked...

[Bart Zorn_1]

Yes, that would be nice.

As is often the case, it would also break things. For example, SYS$MANAGER:TCPIP$CONFIG.COM uses this "feature" ...

Regards,

Bart Zorn

[Wim Van den Wyngaert]

Willem,

Just use
"This is " + bagger

The syntax you use has for me the advantages :
- auto conversion int to str
- auto test on existance

But be careful :
$ bagger=1
$ write sys$output "This is " + bagger
1

Wim

[Jon Pinkley]

If they changed the behavior now, it would cause things to break.

I believe it existed before the f$type lexical existed.

I have seen code that uses the behavior as a shortcut, whether it is is good practice or not.

$ if "''f'" .eqs. "" then f = "something"

instead of

$ if f$type(f) .eqs. "" then f = "something"

If they do change it, they will have to provide a way to change the value of the checking, for example with a new qualifier to set symbol /noundefined_substitution.

This behavior has been there for at least as long as binary valued symbols have been.

Prior to V3? the following was not valid DCL
$ A = 3

Perhaps Hoff can tell us, although I didn't see anyting explicitly about this behavior in "Writing Real Programs in DCL, Second Edition"

Jon

[Willem Grooters]

I'm not the advocate of change here - being well aware that things do break when behaviour changes, especially when it has been this way for such a long time :). However, an additional mode of operation would be a good thing, similar to SET (NO)VERIFY / F$VERIFY(), perhaps?

[Robert Gezelter]

Willem,

An interesting idea, however, there are some limitations if one needs to preserve the previous "classic" behavior.

A mode setting would be useful, but the question is then how to handle nested command procedures. In some cases, they may be implicit through the use of DCL symbols.

I am not saying that it could not be done, but that care in the definition is needed to preserve the idiosyncratic, but functioning use of the anomaly in the existing code base.

It would be interesting to know if the behavior is a intentional feature, or whether it is an "enshrined" quirk.

- Bob Gezelter, http://www.rlgsc.com

[Wim Van den Wyngaert · Accepted solution]

To get equivallent working you can replace
s="Dit is ''bagger'"
by
s="Dit is ''f$str(bagger)'"
Works as before and generates errors when bagger doesn't exist. Can be changed without risk using eve learn.

Wim

[Willem Grooters]

@Bob:
"Similar", not "equal to".
It's just an idea. No more than that. I never said implementation would be easy or without limitations. Some thought:
As you will undoubtably know, one SET VERIFY will effective in called procedures until SET NOVERIFY is encountered. The same couls apply to such a mode switch on symbols. There is another issue, though: the difference between local and global symbols, what to do if SET SYMBOL is encountered?
@Wim:
Good thought, for strings.

BTW: I just tried the arithmatic functions, they all seem to fail if a symbols does not exist:

DKWS38>x=bagger-1
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\BAGGER\
DKWS38>x=bagger*1
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\BAGGER\
DKWS38>x=bagger/1
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\BAGGER\
DKWS38>x=bagger+1
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\BAGGER\
DKWS38>

that's what I would expect in string handling as well.

[Jon Pinkley]

I would give Wim Van den Wyngaert 10 points for his s="Dit is ''f$str(bagger)'" suggestion.

It will do exactly what you asked for (but does require code changes).

However, I don't agree it is without risks, as the problem can now become that s is undefined, where it would not have been before.

I know of no way using existing DCL to flag it as an informational and to go ahead and treat the undefined sysmbol as if it were a symbol with a null value (the current behavior).

Without going through the code with preceding each line that has symbol substitution within quotes, to do something like

$ if f$type(bagger) .eqs. "" then write sys$output "BAGGER is undefined, being treated as null string"
$ s = "Dit is ''bagger'"

I know of no shortcuts. At least Wim's would be easier to do with a learn sequence.

RE:"BTW: I just tried the arithmatic functions, they all seem to fail if a symbols does not exist: ... that's what I would expect in string handling as well."


$ abc = bagger + "abc"
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\BAGGER\
$

It is only in quoted substituion that the undefined symbol is trated as if it were a null string.

And I do beleive that is the original way of testing for an undefined symbol before the f$type lexical function existed. But I can't find any reference to back that up.

[Wim Van den Wyngaert]

...for strings...

???

This also works for numerical values. The f$str will convert the symbol to string just as the double quote would do.

Wim

[Jon Pinkley]

Wim,

He must have been responding to your first suggestion of

Just use
"This is " + bagger

Jon

[Willem Grooters]

This issue surfaced because there was a typo in a large DCL procedure:

$ lInstantie = "VO85A"
...
$ OutString = "This is version ''lInstatie'"

Mind the missing "n" in the second line....

In large, and/or dense DCL code, such a typo is very easily overlooked - it took some time before it was found: by accident!
Of course - defensive programming would require the use of F$STRING or other methods to be sure symbols were properly defined. But what programmer would use it for symbols defined, prepared and used internally only?
("Dicipline, MR. G, is the answer to your question". I know :D) but I still would like some help by DCL)

[John Gillings]

This is NOT an anomoly, it is intentional, documented behaviour.

See OpenVMS User's Manual, Section "12.13.5 Undefined Symbols". It's true of all versions of OpenVMS past and will remain so in all future versions (so get used to it!).

I'd expect any serious DCL programmer to be very familiar with all of Chapter 12 "Defining Symbols, Commands, and Expressions" and especially Sections "12.12 Understanding Symbol Substitution" and "12.13 The Three Phases of Command Processing".

There are many uses for this behaviour, the simplest being implementing flags with symbol existence:

$ IF "''flag'".NES."" THEN ...

This will be FALSE if the symbol is undefined or null and TRUE if non-null.

Another use is for code paths where you can accept a preexisting value, or define a default without having to code an explicit test for existence:

$ IF "''outfile'".EQS."" THEN outfile="DEFAULT.DAT"

A less obvious mechanism is to avoid to initialising a loop variable. This is especially useful to reduce the number of labels required for loops in subroutines.

For example:

$ mysub:
$ item=F$ELEMENT('c'+0,delim,string)
$ ... do something with item
$ c='c'+1
$ IF c.LTS.limit THEN GOTO mysub
$ RETURN

A somewhat contrived example, and a bit dangerous because you don't know the value of c at the start. Perhaps a better example is a self initialising mechanism for generating F$SEARCH stream IDs to guarantee streams are unique across the process:

$ ssid=='ssid'+1
$ loop1: f1=F$SEARCH(filespec,ssid)

(note global symbol definition).

Since DCL is interpretive, minimising labels and eliminating additional checks or commands can improve performance.

There are many options for symbol substitution in DCL. See section 12.12 referenced above. To write good quality, defensive code, you need to be especially careful what mechanism you choose in a particular context. In general, it's usually safer to write:

$ s="quoted string" + symbol

than

$ s="quoted string ''symbol'"

UNLESS you want the code to substitute a null if is undefined. The "+" mechanism also avoids potential problem if contains quotes. Consider:

$ symbol=""""
$ s="quoted string ''symbol' xxx"
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
\XXX\
$ s="quoted string" + symbol + "xxx"
$ show sym s
S = "quoted string"xxx"

In most languages, the architect has carefully chosen and documented rules. Most of them have very good reasons for existing, especially in a language as mature as DCL. In general, rules have good points and bad points (and which is which can vary depending on circumstance). DCL is no exception. The secret is to understand and exploit language rules to your advantage, rather than work against them and complain.

Please make sure you've familiarised yourself with the language definition before claiming a bug or anomoly.

[Willem Grooters]

John:

I know it all (well, most). I don't doubt the intentions, and sure, it will have a reason. Nevertheless, the absence of detection of undefined symbols in some string assignments compared to other contexts is different, no matter it's intentional and well documented - hence "Anomaly". It doesn't mean it's wrong or bad. At least, it is documented. That's good.
The consequence however is that a simple typo in symbol name may now go undetected, so will an now symbol that would normally be defined that is - for some reason) undefined - causing a whole set of applications fail to start or run properly, and finding out what causes it can be a hell of a job, if possible at all. Not just in development of test - it can happen in a production environment as well.

Therefore, it would be a nice thing that undefined symbols were signalled in these assignments - on request only, of course.

Alas, DCL lacks that ability, built in many, if not all compilers.

One should just be aware. (and hopefully, there will be such facility some day :))

[Jon Pinkley]

I agree some type of DCL debugger would be nice, and the ability to see symbol substitution in action, along with the "phase" and scope (global,local+depth).

If you have ever worked with a Cisco router, they have debug built in for all kinds of things, for example, if you use the commands: (NOTE WELL donâ t do this on a production router unless you know what you are doing! It is show here as an example of how cisco debug output looks, as a one possible way DCL substitution tracing could be handled, although with 8K symbols, it could be quite verbose :-)

# term mon
# debug ip nat detailed
IP NAT detailed debugging is on
3w6d: NAT*: i: tcp (172.30.205.81, 2283) -> (192.168.122.22, 23) [14207]
3w6d: NAT*: s=172.30.205.81, d=192.168.122.22->192.168.221.22 [14207]
3w6d: NAT*: o: tcp (192.168.221.22, 23) -> (172.30.205.81, 2283) [20258]
3w6d: NAT*: s=192.168.221.22->192.168.122.22, d=172.30.205.81 [20258]
#undeb all
All possible debugging has been turned off
#

You will see the address translations in action; before and after they are applied to source and destination addresses. It is similar in output to what set watch file produces.

And although these are substitutions are documented, I am not as sure as John G that they were all intentional behaviors, there are some extremely bizarre interactions. Can they be exploited? Possibly. Should the default behavior change? No. I am quite sure if it was being redone as a clean slate project, it would be done differently.

Unfortunately, I don't think much will change in DCL, at least not at the low level. If you have ever looked at the code written by D Cutler in Macro, it is not easy to follow for someone that isn't very familiar with Macro.

Perhaps Guy will chime in. His name is sprinkled through the source.

Jon

[John Gillings]

Jon,

> I am quite sure if it was being redone
>as a clean slate project, it would be done
>differently.

I beg to differ. This behaviour is definitely deliberate. There are numerous things that can be done with the existing implementation that could not be done in a strict "IMPLICIT NONE" type environment. Understanding the rules, you can decide at the time you make a reference to a symbol which behaviour you want.

If you're worried about typos use 3rd phase references (always fail) rather than first phase (substitute null string). It's always possible to choose, for example, use concatenation rather than substitution. Alternatively, use F$FAO to format strings, rather than quoted strings with double apostrophe substitution of symbols.

DCL is interpreted and very loosely typed. This has advantages and disadvantages. One of the disadvantages is the lack of debugging tools - many of the debugging techniques of compiled languages simply aren't possible. For example, no compiler could check for definitions of symbols referred to like this:

$ P'i'=P'i'+1

Back in early V5 days there was some support added in DCL to make it stricter, as a debugging aid (for example, forcing all verbs and qualifers to be spelled out completely). There are remnants which can still be accessed today, but they no longer work. I believe the project was dropped because it broke far too many things to be useful.

If you want stronger typing, there are plenty of other languages available, both compiled and interpreted.

Lamenting that langage X isn't like language Y is only productive if a change to "fix" the complaint is peripheral enough to be made without serious consequences. In this case we're talking about fundamental behaviour of a foundation object within the language. That this question was asked at all indicates a misunderstanding of the documented rules.

>Perhaps Guy will chime in. His name is
>sprinkled through the source.

No disrespect to my good friend Guy, but I think you'll find most of the fundamental design decisions in DCL were made before he was born!

[Willem Grooters]

John,
[quote]
Lamenting that langage X isn't like language Y is only productive if a change to "fix" the complaint is peripheral enough to be made without serious consequences.
[/quote]
You missed my point. This is NOT a lamentation on shortcomings of DCL. I just signalled a situation where this behaviour can cause a problem, potentially UNNOTICED until damage is done:

$ P'i'= P'i'+1

Where P is already defined as a symbol and i is undefined, P's value will be 1 after execution of the assignment.

UNNOTICED is the keyword in this.
It would only be a nice thing if DCL would have a facility to signal undefined symbols in these assignement, _on_specific_request_, of course, so exsiting DCL code would not break.