Problem with Linker Options file on Alpha

[John McL]

I am writing a software tool to enable logical names to be deassigned using a wildcard specification for the logical. Let me assure you this will only apply to user, super and exec logicals. The aim is to gather all the names before starting the deassign phase because trying to remove non-existent logicals is better than accidentally deleting new logicals.

The "proof of concept" is working fine and all logicals, both private and shared, can be listed. I have a problem though with accessing address LNM$AL_HASHTBL.

It's no problem with linking /SYSEXE on our Alpha, but our automated "build" procedures use a linker options file and this method fails.

When the linker options file contains
ALPHA$LOADABLE_IMAGES:SYS$BASE_IMAGE.EXE/SHARE I get about 50 multiply defined symbols (but no problem with LNM$AL_HASHTBL). If I append /SELECTIVE to the above line LNM$AL_HASHTBL is undefined. (The attached doc has more details.)

Why can't I get a clean link with the options file? What am I doing wrong? How do I correct it?

[John Gillings]

John,

I think the problem is a clash between entry points in the "real" DECC$SHR shareable image, and the kernel mode hacked down CRTL in SYS$BASE_IMAGE. Without seeing your full LINK command, I don't know which version your code will call - but I'm guessing it's the SYS$BASE_IMAGE ones because the DECC$SHR symbols are the MULDEF. See the linker map for proof.

Your options file works fine from a MACRO32 program, presumably because it doesn't link against DECC$SHR, and therefore doesn't see any duplicates. If I add DECC$SHR to the link, I get your MULDEF errors.

/SELECTIVE doesn't help because you're linking against a shareable image. You get all or none of the symbols. I'm guessing the reference to LNM$AL_HASHTBL is processed after SYS$BASE_IMAGE, so /SELECTIVE causes it to be skipped entirely (no references), hence UDFSYM.

Simple test, try options files:

test1
SYS$LOADABLE_IMAGES:SYS$BASE_IMAGE/SHARE
SYS$SHARE:DECC$SHR/SHARE

test2
SYS$SHARE:DECC$SHR/SHARE
SYS$LOADABLE_IMAGES:SYS$BASE_IMAGE/SHARE

test1 is equivalent to what you're doing now, since DECC$SHR is picked up at the end of the list, by default from IMAGELIB. MULDEFs are reported against DECC$SHR.

test2 processes DECC$SHR first, so the MULDEFs are reported against SYS$BASE_IMAGE.

Since the shareable images are monolithic there's no way around the warnings if you link against both images. There is a clash in symbol names, so you will get MULDEF. Linking against DECC$SHR first will give you a correct image because you'll be calling the "real" DECC$SHR routines, not the kernel ones.

Changing languages will eliminate DECC$SHR, but if that's not feasible another option is to get the address dynamically, at run time, rather than linked via the options file:

stat=LIB$FIND_IMAGE_SYMBOL('SYS$BASE_IMAGE','LNM$AL_HASHTBL',lnm$al_hashtbl,'SYS$LOADABLE_IMAGES:')

So now you won't need the options file at all.

Note you'll get the same behaviour on Itanium, and the same fix.

[Hoff]

Some reading on duplicate symbols with C kernel-mode code:

http://labs.hoffmanlabs.com/node/769
http://labs.hoffmanlabs.com/node/273

[John McL]

John G, I'm following up your comments. Flipping the order of those lines in the linker option file generated the multiply defined symbol messages and as you say, the image mentioned in the messages swapped around.

I'm currently working on getting LIB$FIND_IMAGE_SYMBOL to work. It's giving me ILLSYMLEN errors at the moment but it's got to be my fault somewhere. I'll award points when I get things working.

In the bigger picture this problem looks likely to occur whenever C code wants to link against SYS$BASE_IMAGE.EXE and the user doesn't want to use LINK/SYSEXE. Doesn't this deserve a mention in the Linker manual?

[John McL]

When I use LIB$FIND_IMAGE symbol in SYS$BASE_IMAGE.EXE in directory SYS$LOADABLE_IMAGES I get the error message:

%LIB-W-ILLSYMLEN, symbol SPARSZ(Ã EXE$C_SYSPARSZ(p ....

PFM$C_BUFCNT has illegal length (89.) in module SYS$BASE_IMAGE

%TRACE-W-TRACEBACK, symbolic stack dump follows (etc.)

When I modified my code to point to other .EXEs in other directories it worked fine.

What's going on?

[x2084]

(All typos and poor formatting are mine: I wrote the answer in a real editor and tried to cut and paste it here. So, no points or negative ones, please.)


>>> 1. With Linker options file containing
ALPHA$LOADABLE_IMAGES:SYS$BASE_IMAGE.EXE/SHARE (which is the documented
equivalent to /SYSEXE).

Where is this documented? The equivalent would be a
ALPHA$LOADABLE_IMAGES:SYS$BASE_IMAGE.EXE/SHARE/SELECTIVE


>>> %LINK-W-MULDEF, symbol DECC$_WMEMSET64 multiply defined
in module DECC$SHR_EV56 file SYS$COMMON:[SYSLIB]DECC$SHR_EV56.EXE;1

The map shows
SYS$BASE_IMAGE ALPHA X9ZK-O2L 0 [SYS$LDR]SYS$BASE_IMAGE.EXE;1
F V1.0 100 SYS$SYSDEVICE:[HARTMUT.L]F.OBJ;2
DECC$SHR_EV56 V7.3-2-00 0 [SYSLIB]DECC$SHR_EV56.EXE;1

The base image exports DECC$_WMEMSET64 and DECC$SHR exports it as well, here
you got you code linked with the one from SYS$BASE_IMAGE.



>>> 2. with Linker options file containing

ALPHA$LOADABLE_IMAGES:SYS$BASE_IMAGE.EXE/SHARE/SELECTIVE (in an attempt to
prevent the above)


%LINK-W-NUDFSYMS, 1 undefined symbol:
%LINK-I-UDFSYM, LNM$AL_HASHTBL
%LINK-W-USEUNDEF, undefined symbol LNM$AL_HASHTBL referenced
in psect $LINK$ offset %X00000148

That's a problem of the order how the you told the linker to process the input
files. With /SELECTIVE you must ensure that that sharable image is processed
AFTER the input file which references LNM$AL_HASHTBL.


>>> Changing languages will eliminate DECC$SHR, but if that's not feasible
another option is to get the address dynamically, at run time, rather than
linked via the options file:

stat=LIB$FIND_IMAGE_SYMBOL('SYS$BASE_IMAGE','LNM$AL_HASHTBL',lnm$
al_hashtbl,'SYS$LOADABLE_IMAGES:')

If that works ("LNM$AL_HASHTBL" is a Double Valued Global Symbol/Version Mask)
then there is another disadvantage: lib$fis doesn't do any checking of GSMATCH
nor the system version array. I recommend to statically link against the base
image.


>>> John G, I'm following up your comments. Flipping the order of those lines in
the linker option file generated the multiply defined symbol messages and as
you say, the image mentioned in the messages swapped around.

$ ty f.c
#include
extern void *LNM$AL_HASHTBL;
void foo(){printf ("%p",LNM$AL_HASHTBL);}
$
$ cc f/stand=rel
$ link tt:/opt/exe=f/map=f/full/cross
sys$share:decc$shr/share
cluster=f,,,sys$disk:[]f
alpha$loadable_images:sys$base_image.exe/share/selective
$


>>> In the bigger picture this problem looks likely to occur whenever C code wa
to link against SYS$BASE_IMAGE.EXE and the user doesn't want to use
LINK/SYSEXE. Doesn't this deserve a mention in the Linker manual?

Looks like a very specific request to me. The recommended way to link against
the base image is to use /SYSEXE. The recommendation is to add this qualifier.

What you need/want to know is sequence of the input file processing by the
linker. There is a section in the manul about this. The sequence is reflected
in the Object Module Synopsis of the map.


>>> When I use LIB$FIND_IMAGE symbol in SYS$BASE_IMAGE.EXE in directory
SYS$LOADABLE_IMAGES I get the error message:

%LIB-W-ILLSYMLEN, symbol SPARSZ(Ã ? EXE$C_SYSPARSZ(p ....

PFM$C_BUFCNT has illegal length (89.) in module SYS$BASE_IMAGE

%TRACE-W-TRACEBACK, symbolic stack dump follows (etc.)

When I modified my code to point to other .EXEs in other directories it worked
fine.

What's going on?

It looks like lib$fis can't handle the "Double Valued Global Symbol/Version
Mask". Which looks OK to me, lib$fis was not designed for that

[Wim Van den Wyngaert]

Nothing to do with your problem but why don't you put your logical names into a logical name table ?

We put all logicals of application X in table X$LOGICAL_NAMES and add that table to the logical search path. When we stop the application we remove the table and remove it from the search path.

Wim

[Hoff]

If we're going to look at the design, then this could be solved in about twenty or thirty minutes with some mildly ugly DCL. SHOW LOGICAL [whatever] /OUTPUT=file.txt or such, and you're off to the races. And the DCL will usually survive upgrades, and usually won't crash the process or the system if something goes "worng".

As for some folks that have already solved this basic sequence, see the LN$SDA widget at

http://eisner.encompasserve.org/~miller/

And see LNMLOOKUP over at

ftp://ftp.process.com/vms-freeware/fileserv/lnmlookup.zip

[John McL]

Harmut, thanks for your detailed response. Here's my reply.

1 - The Linker manual says that the equivalent to /SYSEXE when using a Linker options file is ALPHA$LOADABLE_IMAGES:SYS$BASE_IMAGE.EXE/SHARE and doesn't mention /SELECTIVE.

2 - Sure linking order seems to be important but if I say $ LINK A, B.OPT/OPT mean that A.OBJ gets processed first and then the contents of linker options file B.OPT? This would mean that there should be no problem. (It is possible that the automated procedures that we use here are merging a number of options files and might be re-ordering things - I'll investigate.)

3 - When you discuss using LIB$FIND_IMAGE_SYMBOL and again near the end of your response you talk about 'LNM$AL_HASHTBL' being Double Valued Global Symbol/Version Mask". What do you mean by this? I understand the last part - which just means global address - but are you saying that there's two addresses of the same name (presumably one in my initial address space and another because of the activation of SYS$BASE_IMAGE.EXE?

[John McL]

To Wim and Hoff...

Wim, nice idea but we're talking about software that has experienced ameobic growth for over 10 years, has about 150 CMS libraries and runs to several million lines of code. The code is very modular but to find and modify the references to certain logicals would be a big effort and every piece of code that accessed these logicals would need to be tested, and when that's done we'd have to move every modified executable into a production evironment that runs 24 * 7. I'm taking the easy approach!

Hoff, yes SHOW LOG/OUTPUT=... and parsing the output file is doable is possible. I've been on VMS for 30 years and seen big improvements in speed but this would still be far slower than accessing the data directly. If the data is present and available then why not use it?. Sure it changed at about 4.7 but these structures sem to have been stable since then. Better yet, SYS$TRNLNM should accept wildcards and return the data that I need, although it might have to forget the normal cycling return and just return a pointer to data block.