Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

Creating a shareable image on I64

SOLVED
Go to solution
Martin Vorlaender
Honored Contributor

Creating a shareable image on I64

All,

when trying to build XPDF and all of its dependancies on OpenVMS I64 8.3, I hit a road block with (at least) XAW and XMU.

On VAX and Alpha, a map file is generated from the object files using LINK/NOSHARE/NOEXE/MAP=map_file option_file_containing_objects

Then, map_file is parsed for psect and symbol names. Out of these a new option file is generated to LINK/SHARE.

Unfortunately, this does not work on I64. I do have another DCL procedure that parses the output of an ANALYZE/OBJECT/SECTION=SYMTAB for symbols. This gives me the global procedure symbols.

Unfortunately, XAW also exports some C structures, which I cannot find in the object analysis output.

Anyone have a hint on how to identify these?

cu,
Martin
17 REPLIES
Hein van den Heuvel
Honored Contributor

Re: Creating a shareable image on I64

Horse and cart question ?

I'm all for automating the generation of sources file when possible and desirable.
I have generated many a function table, prototype module, and so on myself, mostly using DCL scripts, but of late more so using perl.

But really IMHO the .OPT file is not a by product but a tool to make sure everything is consistent.
So go ahead, generate it once (on Alpha!), closely verify it for excess symbols which should not be exposed, and then bless the darn thing and stick it in the source distribution.
From there onwards, maintain it like any other source file. This .OPT file should possibly not be seen as an annoyance, but as a helper file.

Just a thought!

Hein.

John Reagan
Respected Contributor

Re: Creating a shareable image on I64

Global data symbols and PSECTs are in the SYMTAB as well. If you can't see them, C didn't export them.

There isn't any other hidden mechanism for global data vs global routines.

John
Richard Whalen
Honored Contributor

Re: Creating a shareable image on I64

If you can find an OpenVMS V8.2 system then you can use the /EXPORT_SYMBOL_VECTOR and /PUBLISH_GLOBAL_SYMBOLS qualifiers to generate an options file. But as was discussed in http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1176046 these are not available on VMS V8.3.

If you intend to make the software available to others, then it would probably be best to find an 8.2(-1) system as it is possible that an image linked on an 8.3 system may not run on an 8.2 system.
John Gillings
Honored Contributor

Re: Creating a shareable image on I64

Martin,

What doesn't work on I64? What's the difference in the map files? What does Alpha list that I64 doesn't? Have you tried /MAP with /FULL and/or /CROSS?

I'd have thought it much quicker, simpler, and easier to parse to dump all the object modules in a library then do a LIB/LIS/NAMES to generate the list of global symbols (though granted, it doesn't list PSECTs - are you really sharing PSECTs between images? uggh! there are much cleaner mechanisms)

As Richard mentioned, for a short time, the linker had /EXPORT_SYMBOL_VECTOR and /PUBLISH_GLOBAL_SYMBOLS, but these were removed for the very reason that what you're doing really isn't a good idea!

First, it's extremely rare that you would really WANT to export ALL global symbols from a set of object modules to build a shareable image. Why clutter up a symbol vector and wear the run time cost of fixups for routines that will never, and should never be called from outside?

Second, even if you generate them, there's no way, even in theory to automatically keep them in a consistent order to form a symbol vector which gives any type of upwards compatibility (and, if you don't care about upwards compatibility, why use shareable images in the first place? much simpler just to link everything together!)

As Hein has suggested, treat the options file as another chunk of source code. By all means generate the list first time, but then maintain it by adding any new symbol table entries at the end. Surely he code can't be volatile enough to make that a serious burden?

A crucible of informative mistakes
mike wagner_4
Occasional Advisor

Re: Creating a shareable image on I64

> ... LIB/LIS/NAMES ...

Lists the names, you very likely want to know if they are procedure or data. At least the linker options expect that. Sure, try and error is always an option.

> ... PSECTs between images? uggh! there are much cleaner mechanisms) ...

Did you ever consider that the compiler may generate them for you and that you may not have much choice?

Xpdf looks like Open Source. If you want to port it you have to take what's there or to convince the non-VMS developers to accept VMS specific code or build procedures, and to maintain them.

On Unix/Linux "exporting everything" is quite common. And it works. On these systems there is not always a penalty for not calling an exported function, the symbols for "everything" can have late binding, they are "fixed up" when they are needed. This doesn't make exporting everything a good thing, but it is cheap. Guess what people pick? In other words, Unix is different.

ANALYZE/OBJECT is the supplied VMS utility to get to the global symbols and the section, which is not necessarily listed as a symbol in the symbol table.

But why aren't we waiting until Martin tells more about his problem?

mw
Hein van den Heuvel
Honored Contributor

Re: Creating a shareable image on I64

>> But why aren't we waiting until Martin tells more about his problem?

I am! :-)

Hein.
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

All,

thanks for all your good thoughts and suggestions.

The VMS build procedures for all the bits and pieces are already there (for Alpha, and partially for VAX; though for earlier versions of at least some of the parts). So I thought the easiest way would be to use them for I64, too.

BTW, a big Thank You to Martin Zinser and Johannes Plass for their work.

@Hein:

That'll probably be the path I will be following. But as John G. mentioned, it's not the way to get an upward compatible image. If I find the time, I'll try to devise a procedure to ensure the order of the symbol vector is kept intact if one already exists.

@John R.:

Of course the global data symbols are in there somewhere - they're just not as readily identifiable as the procedures with their "Symbol Type STT_FUNC"/"Symbol Binding STB_GLOBAL". At least not to me...

@Richard:

Sounds very interesting. I'll look into setting up an 8.2-1 system on another disk of the rx2600.

When I'm back at work I'll post more details, as I don't have an Integrity to play with at home (yet ;-).

cu,
Martin
Willem Grooters
Honored Contributor

Re: Creating a shareable image on I64

Just an idea, since you are on Itanium: Use the Obj2IDL tools that comes with the current version of WSIT. You'll need to build your mdoules /DEBUG, and that will pass a lot of info in the ELF/DWARF format which is scanned by this program - and it will create an XML file containg all the global synmbols you need. The work will be to extract that data...

(Just deducting my limiets knowlegde and expierence with the product)

WG
Willem Grooters
OpenVMS Developer & System Manager
John Reagan
Respected Contributor
Solution

Re: Creating a shareable image on I64

Most global data is STT_OBJECT/STB_GLOBAL with a non-zero "bound to section" field. However, depending on C qualifiers, 'global' data is presented as PSECTs with the matching name.

Global routines are STT_FUNC/STB_GLOBAL with a non-zero "bound to section" field.

Global PSECTs (COMMON blocks, exported C structures.) produce a separate section with the appropriate name. You'll see that in the SECTION SUMMARY. It will be marked Gbl or GblOvr depending on if it is a COMMON block or some C global that uses PSECT naming.

Also, in the symbol table, you'll find an unnamed symbol STT_SECTION/STB_LOCAL that is bound to the Gbl section.

I wrote a little Pascal program:

module m;
var x : [global] integer;
var y : [common] integer;
[global] procedure z; begin end;
end.

and looked at the ANAL/OBJ/SECT=SYMTAB output
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

@Willem:

VERY interesting idea! I'll have a look at it.

@John:

Thanks a bunch for the confirmation. I guess the STT_SECTION/STB_LOCAL confused me, when I was looking for STT_OBJECT/STB_GLOBAL sections instead.

So for those global data symbols, the "global" attribute really is hidden in the section flags, which are only visible in the section summary.


Am appending an example object analysis file for reference.

cu,
Martin
Hoff
Honored Contributor

Re: Creating a shareable image on I64

IIRC, I posted up a symbol-export hack on Freeware V8.0, here:

http://mvb.saic.com/freeware/freewarev80/libxml2-2_6_24/

Buried in there, you'll find a DCL procedure that has all the problems mentioned earlier. Caveats are listed within the comments.

This hack might serve as a variant starting point for your needs. IIRC, what I was doing in this hack was looking at patterns in the source code.

Though what you've already got may work as well for your needs.

Having worked in this area on many occasions, access to the compiler ANA record structures and the related analysis file routines in use underneath DECset can be quite useful. I would hope these could be made semi-public by HP, akin to how the debugger record structure definitions were released. eg: http://mvb.saic.com/freeware/freewarev50/debug/

BTW, there are some decent C parsers around now, so there is another and quite different approach available; I'd probably start with the LLVM front-ends here and not with GCC.

Stephen Hoffman
HoffmanLabs LLC
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

Hoff,

I think I found it - it's the one that says "Do not use this command procedure! Ever!" and goes on to name it "a monstrous and exceedingly ugly hack", right? :-)

Despite the comments, IMHO it's a lovely idea to use LIBRARY/CROSS to locate all relocatable symbols in an object library, the data symbols being the ones with a value of 0...

Thanks for the hint!

cu,
Martin
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

Thanks for all your suggestions.

I have put aside all the hacks, and instead extended elf.c from the GNV source distribution (suggested by Michael Moroney in http://forums12.itrc.hp.com/service/forums/questionanswer.do?threadId=1182334 - thanks, Michael!) to give me all the DATA (i.e. global sections) and PROCEDURE global names that I need.

For reference I have appended a GNU diff file that contains the changes I made.

cu,
Martin
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

So now I learnt that you can't attach something to a closing message...

See above.
mike wagner_4
Occasional Advisor

Re: Creating a shareable image on I64

Cool, so you declare a global PSECT as DATA in a SYMBOL_VECTOR option and that works(?)!

mw
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

If you CC the sources /EXTERN=COMMON/SHARE, then yes, it seems so (don't ask me why the last porter of XMU/XAW did that - probably it has something to do with VAX C compatibility).

As this causes so much hassle, I'm thinking of changing the rules for I64...

cu,
Martin
Martin Vorlaender
Honored Contributor

Re: Creating a shareable image on I64

After removing /EXTERN=COMMON/SHARE from the XMU and XAW3D description files (because with them, I can't tell global data symbol definitions from references), and /NAMES=AS_IS from the FreeType make command files (because it was the easiest way ;-), I've now got a working copy of XPDF.

Thanks everybody! You learn something new every day.

cu,
Martin