URGENT - Logical Names disappearing mysteriously

[Phillip Thayer]

I have a problem with out systems here since yesterday that I cannot figure out. Some of out group level logical application names are being deassigned somehow. Here are the symptoms:

1. The system is running along happy as can be and sudennly some of the lower level group logical names are no longer defined. For example:

"UTL_EXE" = "APPLICATION_DISK:[H_H.COMMERCIAL.UTL.EXE]"
"APPLICATION_DISK" = "COMM_APP_DISK" (LNM$GROUP_000400)
1 "COMM_APP_DISK" = "DSA1:" (LNM$SYSTEM_TABLE)

When we start having problems I do a SHOW LOGICAL on COMM_APP_DISK and I get the error %SHOW-S-NOTRAN, no translation for logical name COMM_APP_DISK which causes the programs to not be found. The root logical names for the data disks are also disappearing which creates major problems for detached processes and batch processes.

2. System logical names are being deassigned or disapperaing somehow. What brought this to my attention was when I tried to do a SHOW CLUSTER and got an error message about the language not being defined. The logical name SYS$LANGUAGE was not defined as well as SYS$ANNOUNCE and SYS$WELCOME. There were several other system level logical names that were not defined as well. All of them were some of the logicals set in VMS$INITIAL-050_LIB.COM command procedure.

I am stumped and do not know what is happening. Our configuration is a cluster between a VAX 4000-705A running VMS V6.2 and an AlphaServer ES40 running VMS V7.3. The cluster was rebooted 10 days ago and we have been running smooth until yesterday afternoon.

I know this is not a standard VMS cluster and that it is not officially supported but we have been running like this for the last year and a half with absolutly no problem. I would love to upgrade the VAX to VMS V7.3 but we are running an application that is now owned by Kodak, they are not supporting the application anymore and will not give up the source code. So I cannot upgrade to a newer version of VMS because I cannot make application changes that are needed. (Like recompile /re-link in a new major operating system version to even run the programs.) All I have are executables.

Any help with the problem of the logical names would be greatly appreciated.

[Wim Van den Wyngaert · Accepted solution]

You can try to put an acl on the logical name table and generate an audit alarm when it is modified.
Then analyze audit to find the process id of the logical name killer.

Wim

[Heinz W Genhart]

Hi Phillip

this sounds very strange to me. I have never seen that logical names in the system and the group table are suddendly missing.

I think there must be something or somebody who deassigns those logicals.

About your kodak application. Did You try to run this application on VMS 7.3. I would expect, that those image are running without to recompile and to relink. (If that is normal usercode and not depending on SYSDEF)

You may see a hopefully small performance degradation, but in most cases the will run


regards

Heinz

[Phillip Thayer]

Wim - I will try the ACL idea and see what I get from that. I totaly forgot about that. Thanks.

Heinz - VMS does not allow you to take an executable from a V6.x system and run it on a V7.x system. You get an error that says something about the image not being compatible. Thanks for the suggestion though.

[Wim Van den Wyngaert]

Check http://h71000.www7.hp.com/openvms/journal/v6/protecting_and_monitoring_openvms.html
in the middle "Example of Logical Name Table Security Auditing".

More simple but try with access delete or write.

Wim (didn't use that myself yet)

[Robert Gezelter]

Phillip,

I agree with the suggestion of instrumenting the system with alarms.

On the images being incompatible with releases later than 6.2, that does indeed sound unusual. Can you be more specific on the precise error message(s) that are appearing.

There are some run-time library issues, but they generally can be circumvented.

- Bob Gezelter, http://www.rlgsc.com

[Phillip Thayer]

Robert, I cannot remember the exact error message. I know is had something to do with the image being built on a different major version of VMS. It has been a while since I tested this on a VAX I temporarily had access to with VMS V7.2. My concern at the time was running an upgrade on a production system, having our primary application not run and having to fall back to VMS V6.2 all in one afternoon.

Phil

[Wim Van den Wyngaert]

Just tried the example. It generates some output. Use disable to deactivate it again. You don't see what is exactly modified.

Wim

[Phillip Thayer]

Wim,

I am not too concerned about what is modified but rather who or what is the culprit that is modifying the logicals. I did the example and it generated a huge number of alarms. I disabled it and now have it enabled for audit logging ONLY without alarming so I can get the data when the problem happens without having the system console beeping like crazy until then. After that I will disable the audit logging.

Phil

[Phillip Thayer]

I have a correction to make to the symptoms I described. The logicals in the group tables are not being affected. The logical I listed COMM_APP_DISK is actually in the system table (LNM$SYSTEM_TABLE). So are all the other logicals that seem to disappear.

Phil

[Phillip Thayer]

I am attaching the output listing from SYSGEN SHOW/ALL. Maybe I am missing something here that I could be causing the system to exhibit this kind of problems.

Phil

[Phillip Thayer]

Another question. If I have a system logical name and I assign a value to the logical, what is the LARGEST value that can be used with that logical name. For example:

DEFINE/SYSTEM 310_THAYERP "logical-name-value"

What is the maximum length of "logical-name-value" that is allowed by the system.

Phillip

[Wim Van den Wyngaert]

http://h71000.www7.hp.com/doc/82final/5932/5932pro_041.html

It is 255.

Wim@home

[Phillip Thayer]

O.K. So, if I have a logical name being created with the LIB$SET_LOGICAL in a program and the value is longer than 255 could it cause the problem I was seeing?

We use a system logical with the users name to track what they are entering in the programs. This was added about a week or so ago. These logical names were not clearing the value out but rather appending the new information onto the end of the logical name. I'm sure that some of them were running over the 255 limit and I am wondering if that could have caused the systems LNM Hash table to create a situation where it overwrote or deleted logicals. Does that sound like a possibility?

Phil

[Wim Van den Wyngaert]

Everything is possible. If you are the first to do that (in almost 10 years 6.2) you might be the bugfinder.

Did you install all patches ? Is the logical visible when using wildcards ? May be do a show logical to a file, a bit later again and do a diff. But what about sys$announce ?

But did the audit stuff report something usefull ?

I would reboot and see if it comes back.

Wim

[Robert_Boyd]

One strategy I've used for going beyond the 255 character limit for equivalence strings is to turn the logical name into a search list.

You can have 128 list elements, so each name could have up to 128*255 characters.

I've extended my use of this further in code that captures output of a pipe and stuffs it into logical name(s) to avoid creating a file.

If it uses up the 128, it bridges to another logical name by making the last list element a pointer to another list.

Please find the DCL version of the code attached here. There is a C version of this program that I wrote long ago, but I only have executable copies of it anymore. (time to rewrite it I suppose).

This DCL version depends on the freeware utility SLMOD (Search List Modifier).

Robert

[Phillip Thayer]

I went into the programs that we had done this with and changed it to initialize the string with a blank when the users starts looking at a new order number. Basically, these logicals are used so that we can see who has what order/account record locked. Our billing application will freeze with a record locked message if someone is looking at an order and has the order or account record locked. Then they call me and I am now able to call the person who hasthe record locked and ask them to exit from the program.

The problem with the long logical names was that it was not initializing the logical to blank when they request a different record. It was appending the record/account number to the value currently in the logical.

The problem has not happened again so the audit has not showed my anything yet. If it does happen again (hopefully it will not) then I will look at that first after I get everything running again.

Phil

[Wim Van den Wyngaert]

Philip,

What is/was the reason why sys$announce was gone ?

Wim

[Phillip Thayer]

Well, since I made the change to the program it has not happened again. Here is one of the reasons I was looking at this logical. Both times I had this happen was at shift change times when production supervisors are checking orders that are currently in production. When I talked to them they all said that they typically go into the program that I had the logical name setting in and display anywhere from 20 to 30 orders before they exit from the program. If they did this it would have extended the logical name value past the 255 limit. Since this was something that was one of the last things changed on the system begore the problem started, it was the likely suspect as the root cause of the problem.

So, I changed the way that this logical is being set and have not had a problem since.

Phil

[Wim Van den Wyngaert]

And sys$announce ?

Wim

[Phillip Thayer]

The only thing I can say about the SYS$ANNOUNCE is that the logicals like SYS$ANNOUNCE, SYS$WELCOME and SYS$LANGUAGE were lost somehow when the hash table blew up on me. Other than that I have no explaination as to how they disappeared.

[Phillip Thayer]

We have found the peice of code that was causing the problem with logical names being wiped out.

Here is the scenerio:

We have a VAX Basic program that creates a logical name to show what account and order is being accessed by the user. This logical name was created by getting the users process name using SYS$GETJPIW with an Item list. Immediately before this code there is a section of code that gets the terminal name of the user using the SYS$GETJPIW with a different item list.

Here is what was causing the problem. If the code that was getting the terminal name returned no value the program was going to the program label directly after the section of code that gets the process name. We have a product called MAGIC that allows a user (usually a developer) to have multiple sessions on a single powerterm window. The way it does this is by creating a subprocess. These subprocesses have a terminal name of FTAx:. If someone is in one of these subprocesses created by MAGIC the SYS$GETJPIW would return a blank value and the value for the logical name, STAT_LOGICAL$, would not be set. In VAX Basic this would make the logical name variable STAT_LOGICAL$ equal 255 Null characters. We were able to reproduce the problem at will and identified the code section in debug.

When this situation happened and the program would later attempt to create or delete a logical name with the uninitialized variable STAT_LOGICAL$ in the system table using LIB$SET_LOGICAL or LIB$DELETE_LOGICAL it would cause the loss of logical names in the system table.

So, the code was changed to not go around the code to get the process name and everything is working fine.

I have attached the snippets of code that caused the problems.

Phew, This was a tough one but we got it figured out.

Phil

[Robert_Boyd]

Still sounds like a failure in the LNM code in the executive part of the system failing to detect bad input, or doing something unexpected with data of an unusual format.

Robert

[David B Sneddon]

Robert,

What is wrong with setting the value of a logical
to 255 nulls?

Dave

[Phillip Thayer]

Robert,

I agree with you on this one. I think that there may be something that is not handling the case where a logical name contains null characters in the logical name.

David,

I don't think it's a problem with the logical name values having nulls but with the name of the logical having nulls.

Phil