Change in .EXE size on rebuild / odd reference to FORRTL in a C/Pascal-coded image?

[Mark_Corcoran]

The company at one of whose sites I'm based, is owned by a sister company of my employer.  When it was sold to the sister company by its previous owner, this included a dump of the source code for the 3 applications my team look after (and probably, other applications that we don't).

I've had occasion to rebuild a few executable images after either adding new features or fixing extant bugs in the source.

This week, after consolidating 4 text files of "tasks to do" - essentially a bucket-load of Post-It notes (other brands are available ;-)) - I've finally got around to rebuilding one executable image that we had problems with two years ago when it kept crashing.

[Actually, it was the Pascal RTL terminating the process because of attempting to access beyond the bounds of an array.

The code attempted to examine a block of elements within the array that pertained to a certain type of logical object, and didn't cater for when the last remaining object of that type was deleted.

It then didn't execute the code to calculate the index position, so the procedure returned whatever happened to be on the stack in the position that was allocated to the variable used to store the result of the calculation.

We worked around it at the time (once I eventually determined what was happening) by adding a placeholder object which created an entry in the on-disk database and the in-memory array copy, allowing the index calculation to work]

The original build procedure that the developers created rebuilds all of the executable images used by the application, but I don't want to have to fix any environmental differences that cause compiler or linker warnings in executable images that I don't actually need to rebuild, so have created per-executable build procedures.

I don't have a specific reason for actually rebuilding the executable image, only that if a similar problem persisten abnormal termination event should occur again in future, I don't want to be scrabbling around trying to create a build procedure whilst we have a production outage (most of the code is written in Pascal, and there's a lot of conditional/positional compilation required to produce Pascal Environment (.PEN) files that are inherited by the .PAS modules that need to be compiled).

A concern has always been that the code we were given, might not match that which was actually used to build the executables that we are running, and I wasn't happy to see that for this particular image, there was a significant decrease in size (the original/current version was 833 blocks, and the new one 724 blocks).

I performed an ANALYZE /IMAGE of both the original and the new one, and part of the reason for the size difference I'd effectively already determined when resolving compiler informational messages and linker warnings…

The way I wrote the build procedures was to examine the modification date of each source and object module and only recompile the source if it was more recent than the object.

This resulted in me getting linker warnings for a few modules, about unknown symbols for C RTL functions.

It took me a while to "see the wood for the trees", but the reason for this was because I wasn't recompiling these modules, and just attempting to link the ones that had been produced on the original development system.

We have got DEC C (or HP C if you insist) installed on our test system, whereas the original developers had VAX C (and then subsequently, DEC C).

As we have DEC C rather than VAX C, the C RTL symbols in SYS$LIBRARY:IMAGELIB.OLB are prefixed with DECC$, hence why the symbols for the (VAX) C RTL functions couldn't be found when trying to link the old VAX C-generated object files.

I then forced compilation of these modules and got compiler informational messages (%CC-I-IMPLICITFUNC) because I copied the CC commands for these modules directly from the original build procedure.

These modules didn't (:-O) #include the relevant C RTL header file, and (I guess) because the developers had been using VAX C, the compiler behaviour was different (changing the CC command to use /STANDARD=VAXC made these "go away", but I have since added #include directives as necessary).

So, back to the size difference of the .EXE between the new and old ones…

I used ANALYZE /IMAGE to compare the old and new ..EXEs, and I've extracted a summarised version of what I would consider the most pertinent information, and attached it (otherwise the alignment goes to pot).

The fact that DECC$SHR is referenced in the new .EXE isn't a surprise - the .C modules were compiled with DEC C rather than VAX C.

A little more perturbing however, is the reference in the old .EXE to FORRTL.EXE - the source modules are C and Pascal, one .CLD file compiled with the Command Definition utility, and one message file compiled with the Message utility.

Even though I would not have expected FORRTL.EXE to be referenced in the first place, its "sudden" disappearance (accounting for most of the size change) is cause for concern (the executable is farily complex, there's no test scripts for it, and I don't have the age of the universe to generate tests (and database content) that could test every conceivable code path;  if the .EXE is suddenly shorter, what else might be "wrong"?).

I've looked at the symbols defined in SYS$LIBRARY:IMAGELIB.OLB for FORRTL, and a SEARCH of all of the original .OBJ files that make up the .EXE don't show any instances of FOR$ (although of course, this presumes that the original .OBJ files we have, are the same as the ones that were linked together to create the .EXE that we have).

It has to be said that the output of ANALYZE/IMAGE doesn't mean a great deal to me, and I'm not sure if there is any way to determine whether or not the "6 references to image 3" in the Reference Fixup section actually means that in the old .EXE, FORRTL could definitely be executed, or if there's conditional code that even with all optimisation enabled, the (presumably VAX C) compiler couldn't determine whether or not the code would ever be executed;  I'm assuming that the symbols reference routines in the Fortran RTL rather than some language-specific global symbol (equivalent to something like vaxc$errno).

I wondered whether or not the VAX C compiler perhaps had some dependency on FORRTL for some reason (the linker ID is the same between old and new, so it can't be the linker that's causing the discrepancy), or perhaps one of the other shareable images that are mentioned, has a dependency on FORRTL (or at least, did on the original developers' development system at the time the .EXE was generated)?

In the Reference Fixups section, the header indicating the number of references to an image, is followed by a longword hexadecimal representation for each reference.

Does anyone know enough about .EXEs to help determine whether or not that information might help detect what/where the references to FORRTL come from (or have any other sage words of advice that might set me on the right path)?

Any help/advice greatly appreciated!

Mark

[H.Becker · Accepted solution]

In the shareable image list of the old image there is no VAXCRTL[.EXE]. That means, the main image was linked with and contains the VAXCRTL object modules. The new image lists DECC$SHR[.EXE], so there are no C-RTL object modules. in the main image. This explains the difference in size.

Referencing a shareable image in a main image doesn't need much disk size. With 6 references to the FORRTL.EXE this should be less than half of a disk block. What caused the FORTL to be included in the main image is not obvious. The easiest way to find out is to look at the cross reference section of the linker map. If you don't have one you can try to reverse the link process: from the fixup, which points to a location in the shareable image, you need to find the associated universal symbol. Shouldn't be so hard. The name may give you a clue.

For Alpha and I64 there is a tool, which does this for you: XPD (eXternal Procedures and Data). I'm not aware of any VAX tool to display that information. As you probably know, VAX shareable images are somehow different as they don't contain a symbol vector, just a transfer vector and universal symbols.

[Mark_Corcoran]

Thanks for the reply, Hartmut.

>The easiest way to find out is to look at the cross reference section of the linker map.
Ah yes, I should perhaps have said that there wasn't one... in the original build-everything procedure, all of the LINK commands specify /NOMAP

IMHO, unless you don't have disk space on the target volume to accommodate them during the build process, not doing /LIST on a compile (and where the relevant language compiler supports it, a /MACHINE_CODE and /CROSS_REFERENCE) and a /MAP (and perhaps /SYMBOL_TABLE) on a LINK is a fallacy that will come back to haunt you later…

If you have original Vaxen/Alpha hardware and buying DEC/Compaq/HP-branded disk drives is not an option (no budget, no space in the system chassis/storage expansion, the only disks supported by the hardware are no longer manufactured and any "new" ones would be second-hand and of questionable integrity/lifespan), then you should at least FTP the .LIS/.MAP/.STB files onto other newer hardware with more/cheaper disk space (or consider moving to emulation on Windows where disks would be cheaper).

>In the shareable image list of the old image there is no VAXCRTL[.EXE]  That means, the main image was linked with and contains the VAXCRTL  object modules. The new image lists DECC$SHR[.EXE] so there are no C-RTL object modules. in the main image. This explains the difference in size.

I didn't show it in what I considered the pertinent information, but the ANALYZE /IMAGE of the old executable showed one image section descriptor:
page count: 93
base virtual address: %X'00000000' (P0 space)
page fault cluster size: default
ISD flags:
(0) ISD$V_GBL 1
(1) ISD$V_CRF 0
(2) ISD$V_DZRO 0
(3) ISD$V_WRT 0
(7) ISD$V_LASTCLU 0
(8) ISD$V_INITALCODE 0
(9) ISD$V_BASED 0
(10) ISD$V_FIXUPVEC 0
(11) ISD$V_RESIDENT 0
(17) ISD$V_VECTOR 0
(18) ISD$V_PROTECT 0
section type: ISD$K_SHRPIC
base VBN: 0
global section major id: %X'01', minor id: %X'000064'
match control: ISD$K_MATLEQ
global section name: "FORRTL_001"

I don't know enough about the format of executable images (and have never really needed to know), but does the "page count: 93" not mean that (in the on-disk) executable image, 93 pages (VAX, ergo 1 page=1 block) are somehow consumed  for this global section (or simply that at run-time, the size of the $CRMPSCed global section will be 93 pages)?

>Referencing a shareable image in a main image doesn't need much disk size. With 6 references to the FORRTL.EXE this should be less than half of a disk block. What caused FORTL to be included in the main image is not obvious.

This is the crux of the size issue, and I'm happy to be able to provide an update!

Yesterday morning when I woke up early on a bank holiday and was considering trying to sleep for longer, my thoughts turned back to the problem.

It occurred to me that for licensing (cost) reasons, the original owners wouldn't have compilers on the production systems, and that to avoid any shareable/RTL image or (OpenVMS version) environmental issues, they might simply copy the object files onto production systems and link the executable images there.

On my laptop I have a CharonVAX instance of an old copy of the system, with the same source/build directory tree, and the same (unmodified) executable image.

I had a look, but I couldn't find any other command procedures that just linked the object files.

However, I created a procedure to do this, including the /MAP and /SYMBOL_TABLE on the LINK command.

Initially, I got errors about the C RTL symbols being unresolved until I modified the LINK command to add SYS$SHARE:VAXCRTL.OLB/LIB

Unsurprisingly, the resulting .EXE matched the same size as the current one, so that confirms your assertion of it being linked with and containing the VAXCRTL object modules - only one test system has the DECCRTL.EXE shareable image (because it is the system on which we have the DEC C Compiler installed), but they all have DECCRTL.OLB and VAXCRTL.OLB

The resulting map file shows that from FORRTL.EXE, the symbols FOR$CVT_D_TE, FOR$CVT_D_TF and FOR$CVT_D_TG are referenced by C$$DDOPRINT, and the symbols FOR$CVT_G_TE, FOR$CVT_G_TF and FOR$CVT_G_TG are referenced by C$$GDOPRINT;  it also references C$DECVT in C$$DDOPRINT

The object module synopsis of VAXCRTL.OLB indicates that C$DECVT was created by VAX MACRO (V5.4-3)

I was going to have said that I wasn't sure what C$$DDOPRINT and C$$GDOPRINT are but that I would bet my bottom $currency_unit that they were probably related to printf(), sprintf() and printf().

A Google search for C$$DDOPRINT and FOR$CVT_D_TG found the following (Tim Sneddon's page for the C$$DDOPRINT, and The Berryman's for FOR$CVT_D_TG):

http://tim.sneddon.id.au/blog/Posts/asprintf_for_OpenVMS

https://www.theberrymans.com/decus/vmsdisasm/DISMINSTR.MAR?httpd=content&type=text/plain;%20charset%3DISO-8859-1

So, I'm happy that the .EXE size is reduced (when recompiling .C modules on a system with DEC C installed, and relinking the resulting object files on the same system) because FORRTL is no longer referenced, and that FORRTL Is no longer referenced because the .C modules were compiled with DEC C rather than VAX C (and for some reason, VAX C is indeed making use of the FORRTL run time, rather than doing it itself (or in Macro-32) – though I suppose, what's the point in making a new wheel when you have a perfectly good round one already?)

>I don't have a specific reason for actually rebuilding the executable image
Actually, there was a reason for rebuilding it, which I forgot about when I got distracted by the size change.  That reason will appear shortly in another (new) post.

Mark

[H.Becker]

I didn't show it in what I considered the pertinent information, but the ANALYZE /IMAGE of the old executable showed one image section descriptor:
page count: 93
base virtual address: %X'00000000' (P0 space)
page fault cluster size: default
ISD flags:
(0) ISD$V_GBL 1
(1) ISD$V_CRF 0
(2) ISD$V_DZRO 0
(3) ISD$V_WRT 0
(7) ISD$V_LASTCLU 0
(8) ISD$V_INITALCODE 0
(9) ISD$V_BASED 0
(10) ISD$V_FIXUPVEC 0
(11) ISD$V_RESIDENT 0
(17) ISD$V_VECTOR 0
(18) ISD$V_PROTECT 0
section type: ISD$K_SHRPIC
base VBN: 0
global section major id: %X'01', minor id: %X'000064'
match control: ISD$K_MATLEQ
global section name: "FORRTL_001" 

I don't know enough about the format of executable images (and have never really needed to know), but does the "page count: 93" not mean that (in the on-disk) executable image, 93 pages (VAX, ergo 1 page=1 block) are somehow consumed  for this global section (or simply that at run-time, the size of the $CRMPSCed global section will be 93 pages)?

Where should this be in the image? At "base VBN:0"? As you probably know VBNs start with 1. So these are NOT 93 pages in your image. This page count is the size of the global section which is mapped from FORRTL.

Try a little (useless) macro program, which just references FOR$CVT_D_TE, for example:

        .ENTRY FOO, 0
        MOVL G^FOR$CVT_D_TE, R0
        RET
        .END FOO

How big will it be? And then try another one which loads SS$_NORMAL into R0. How big is that and what's the difference in size?

[Mark_Corcoran]

Although it's one month short of 30 years I've been working with VMS, I think you credit me with more low-level knowledge than I possess (certainly as regards .EXE formats) :-D

>How big will it be? And then try another one which loads SS$_NORMAL into R0. How big is that and what's the difference in size?
Harumph.  You're going to make me work for this!

I couldn't speculate how big the resulting .EXE would be, and it's been a long time since I've really done any Macro-32 - I had to MACRO and LINK the resulting .OBJ

4 blocks for both .EXEs, where the FORRTL-referencing one has an image section descriptor with global section name: "FORRTL_001" and a page count of 69.

I was going to have said that that ANALYZE /IMAGE can only be understood by those who already understand it (with no path to enlightenment for those that don't), but after taking a quick glance at the DCL Dictionary A-M, and reading between the lines of what it says, then yes, it refers to the amount of memory to be allocated, not the amount of memory (disk space) consumed by the image section on disk.

So, the difference in size is mostly because certain modules from VAXCRTL.OLB are no longer linked into the .EXE (although there will be some other ancillary size differences - fewer image section descriptors, etc.)

As only a certain number of C RTL functions are used, not all of the object modules from the .OLB were linked into the .EXE in the first place, hence why it's obviously not simply a size difference of (original .EXE size - size of VAXCRTL.OLB) - and in any case, I would guess that there would be some overhead of library management in the .OLB file that wouldn't be included when object modules from it are linked into the .EXE

Mark