Creating a shareable image on I64

[Martin Vorlaender]

All,

when trying to build XPDF and all of its dependancies on OpenVMS I64 8.3, I hit a road block with (at least) XAW and XMU.

On VAX and Alpha, a map file is generated from the object files using LINK/NOSHARE/NOEXE/MAP=map_file option_file_containing_objects

Then, map_file is parsed for psect and symbol names. Out of these a new option file is generated to LINK/SHARE.

Unfortunately, this does not work on I64. I do have another DCL procedure that parses the output of an ANALYZE/OBJECT/SECTION=SYMTAB for symbols. This gives me the global procedure symbols.

Unfortunately, XAW also exports some C structures, which I cannot find in the object analysis output.

Anyone have a hint on how to identify these?

cu,
Martin

[Hein van den Heuvel]

Horse and cart question ?

I'm all for automating the generation of sources file when possible and desirable.
I have generated many a function table, prototype module, and so on myself, mostly using DCL scripts, but of late more so using perl.

But really IMHO the .OPT file is not a by product but a tool to make sure everything is consistent.
So go ahead, generate it once (on Alpha!), closely verify it for excess symbols which should not be exposed, and then bless the darn thing and stick it in the source distribution.
From there onwards, maintain it like any other source file. This .OPT file should possibly not be seen as an annoyance, but as a helper file.

Just a thought!

Hein.

[John Reagan]

Global data symbols and PSECTs are in the SYMTAB as well. If you can't see them, C didn't export them.

There isn't any other hidden mechanism for global data vs global routines.

John

[Richard Whalen]

If you can find an OpenVMS V8.2 system then you can use the /EXPORT_SYMBOL_VECTOR and /PUBLISH_GLOBAL_SYMBOLS qualifiers to generate an options file. But as was discussed in http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1176046 these are not available on VMS V8.3.

If you intend to make the software available to others, then it would probably be best to find an 8.2(-1) system as it is possible that an image linked on an 8.3 system may not run on an 8.2 system.

[John Gillings]

Martin,

What doesn't work on I64? What's the difference in the map files? What does Alpha list that I64 doesn't? Have you tried /MAP with /FULL and/or /CROSS?

I'd have thought it much quicker, simpler, and easier to parse to dump all the object modules in a library then do a LIB/LIS/NAMES to generate the list of global symbols (though granted, it doesn't list PSECTs - are you really sharing PSECTs between images? uggh! there are much cleaner mechanisms)

As Richard mentioned, for a short time, the linker had /EXPORT_SYMBOL_VECTOR and /PUBLISH_GLOBAL_SYMBOLS, but these were removed for the very reason that what you're doing really isn't a good idea!

First, it's extremely rare that you would really WANT to export ALL global symbols from a set of object modules to build a shareable image. Why clutter up a symbol vector and wear the run time cost of fixups for routines that will never, and should never be called from outside?

Second, even if you generate them, there's no way, even in theory to automatically keep them in a consistent order to form a symbol vector which gives any type of upwards compatibility (and, if you don't care about upwards compatibility, why use shareable images in the first place? much simpler just to link everything together!)

As Hein has suggested, treat the options file as another chunk of source code. By all means generate the list first time, but then maintain it by adding any new symbol table entries at the end. Surely he code can't be volatile enough to make that a serious burden?

[mike wagner_4]

> ... LIB/LIS/NAMES ...

Lists the names, you very likely want to know if they are procedure or data. At least the linker options expect that. Sure, try and error is always an option.

> ... PSECTs between images? uggh! there are much cleaner mechanisms) ...

Did you ever consider that the compiler may generate them for you and that you may not have much choice?

Xpdf looks like Open Source. If you want to port it you have to take what's there or to convince the non-VMS developers to accept VMS specific code or build procedures, and to maintain them.

On Unix/Linux "exporting everything" is quite common. And it works. On these systems there is not always a penalty for not calling an exported function, the symbols for "everything" can have late binding, they are "fixed up" when they are needed. This doesn't make exporting everything a good thing, but it is cheap. Guess what people pick? In other words, Unix is different.

ANALYZE/OBJECT is the supplied VMS utility to get to the global symbols and the section, which is not necessarily listed as a symbol in the symbol table.

But why aren't we waiting until Martin tells more about his problem?

mw

[Hein van den Heuvel]

>> But why aren't we waiting until Martin tells more about his problem?

I am! :-)

Hein.

[Martin Vorlaender]

All,

thanks for all your good thoughts and suggestions.

The VMS build procedures for all the bits and pieces are already there (for Alpha, and partially for VAX; though for earlier versions of at least some of the parts). So I thought the easiest way would be to use them for I64, too.

BTW, a big Thank You to Martin Zinser and Johannes Plass for their work.

@Hein:

That'll probably be the path I will be following. But as John G. mentioned, it's not the way to get an upward compatible image. If I find the time, I'll try to devise a procedure to ensure the order of the symbol vector is kept intact if one already exists.

@John R.:

Of course the global data symbols are in there somewhere - they're just not as readily identifiable as the procedures with their "Symbol Type STT_FUNC"/"Symbol Binding STB_GLOBAL". At least not to me...

@Richard:

Sounds very interesting. I'll look into setting up an 8.2-1 system on another disk of the rx2600.

When I'm back at work I'll post more details, as I don't have an Integrity to play with at home (yet ;-).

cu,
Martin

[Willem Grooters]

Just an idea, since you are on Itanium: Use the Obj2IDL tools that comes with the current version of WSIT. You'll need to build your mdoules /DEBUG, and that will pass a lot of info in the ELF/DWARF format which is scanned by this program - and it will create an XML file containg all the global synmbols you need. The work will be to extract that data...

(Just deducting my limiets knowlegde and expierence with the product)

WG

[John Reagan · Accepted solution]

Most global data is STT_OBJECT/STB_GLOBAL with a non-zero "bound to section" field. However, depending on C qualifiers, 'global' data is presented as PSECTs with the matching name.

Global routines are STT_FUNC/STB_GLOBAL with a non-zero "bound to section" field.

Global PSECTs (COMMON blocks, exported C structures.) produce a separate section with the appropriate name. You'll see that in the SECTION SUMMARY. It will be marked Gbl or GblOvr depending on if it is a COMMON block or some C global that uses PSECT naming.

Also, in the symbol table, you'll find an unnamed symbol STT_SECTION/STB_LOCAL that is bound to the Gbl section.

I wrote a little Pascal program:

module m;
var x : [global] integer;
var y : [common] integer;
[global] procedure z; begin end;
end.

and looked at the ANAL/OBJ/SECT=SYMTAB output