Unable to link due to attribute SHORT - PASCAL

[stephenbrayshaw]

During the linking process it is failing due to the SHORT attribute conflicting with the other attributes.

This is on Integrity with 8.4 and Pascal 6.1.

I am porting from Alpha 8.2 with Pascal 5.6.

How can I remove the SHORT attribute from this module / section?

The logs are below:

%ILINK-W-CONPSCATT, attribute SHORT conflicts with SHR; set to NOSHR
section: CD_COMMON
module: CD_COMMON$START
file: SYS$SYSDEVICE:[DM9415.WORK.OBJECT]CD_COMMON$START.OBJ;1
%ILINK-W-CONPSCATT, attribute SHORT conflicts with SOLITARY; set to NOSOLITARY
section: CD_COMMON
module: CD_COMMON$START
file: SYS$SYSDEVICE:[DM9415.WORK.OBJECT]CD_COMMON$START.OBJ;1
%ILINK-W-CONPSCATT, attribute SHORT conflicts with SHR; set to NOSHR
section: CD_COMMON
module: CD_COMMON
file: SYS$SYSDEVICE:[DM9415.WORK.OBJECT]CD_COMMON.OBJ;1
%ILINK-W-CONPSCATT, attribute SHORT conflicts with SOLITARY; set to NOSOLITARY
section: CD_COMMON
module: CD_COMMON
file: SYS$SYSDEVICE:[DM9415.WORK.OBJECT]CD_COMMON.OBJ;1

[H.Becker]

Out of the top of my head - without checking as I don't have access to a Pascal compiler on I64 ...

The linker warns, removes the conflicting attributes and creates an image: so "Unable to link" may not be exactly what happened but it may be that the result is not what you want.

The SHORT attribute was/is set by the compiler. For data, the compiler places small data items into that section: integers, etc. If you do an ANALYZE/OBJECT you should see the attribute "Sho" for the corresponding section. Why there are other conflicting attributes set is not obvious from the posting. I assume this was done intentionally to share this data and to isolate this data by giving it some/an own page/s with the solitary attribute.

The compiler generates code to address data in SHORT data. To get rid of the SHORT attribute for this section in the object module the compiler has to use a different section type and to generate different code. That is, you need to tell Pascal not to create a SHORT data section for this data (and you can't change that later with a linker option).

So the questions are, how was SHR and SOLITARY specified in the first place and what's in the Pascal manual/help to let the compiler use something different than SHORT data for this data.

[Hoff]

The linker is expected to create short segments.  Two, by default.  That's normal.

Are you working with a COMMON here, as might be implied by the object name and some of the attributes shown?  (Best to use global sections here, if that's at all possible.)

As a start, please post the LINK command and the options file in use.  If you expurgate parts of that, please do so consistently.  (This is headed toward an investigation of the PASCAL and LINK commands, the options file, and a map from the Linker, too.)

Alternatively, ring up HP support for a look at this.

[H.Becker]

>>> The linker is expected to create short segments. Two, by default. That's normal.

Yes, more or less, but segments are not the reported problem, here, except that the linker can't create any segment out of (short data) sections with different/incompatible section attributes. And that's what the linker reports, no matter how these attributes made it to the linker. They very likely were in linker options: when processing the options the linker only uses the name of the section to which the attributes from the option are assigned. So for a short data section with the name CD_COMMON one can try to add the SHR and SOLITARY (which may be the same option, which  was used and worked on Alpha). Later, when creating the short data segments, the linker collects all short data sections (compiler and linker generated ones) which then must have compatible/correct attributes. That's the time to inform about conflicting attributes.

>>> As a start, please post the LINK command and the options file in use. ...

As briefly mentioned, there is nothing the linker can do to get rid of the SHORT attribute, which obviously is in the object module and was set by the compiler: the generated code depends on this, that is the code depends on the GP and short offsets. It is more interesting to know what the contents of CD_COMMON is, as declared/defined in the sources. Usually variables with size up to 8 bytes are placed in short data sections. Maybe the compiler has a qualifier or language directive to avoid creating the short data section for CD_COMMON. It would/could be something like specifying the maximum size of variables to be placed there: 0 would be the right value to avoid creating a short data section at all. The other "language" option would be to have these "short" variables in bigger data structures so that the compiler - by default - will not place them into a short data section. But that's more or less speculation on the purpose and content of CD_COMMON.

[stephenbrayshaw]

The area is used as shared memory and is full of arrays with between a few hundred to 4 hundred thousand elements (size is hard coded).

each array is global, volatile and has psect options which are PSECT_ATTR=CD_COMMON, CON,GBL,SHR,SOLITARY

The link command

LINK /NOCONTIGUOUS -
/EXECUTABLE = EXE:'FILENAME' -
/MAP = MAP:'FILENAME' /FULL -
/SYSLIB -
/TRACEBACK -
'P2' -
SRC:UT_LINK$OPTION/OPTION, 'SPECIAL_OPTION' -
OBJ:'FILENAME', -
OBJ:CD_COMMON$START, -
OBJ:CD_COMMON, -
OBJ:CD_COMMON$END, -
OBJ:DAI_LIB/LIBRARY, -
SRC:MQ_LINK_OPTIONS/OPTIONS

If the issue is with the compiler then how can I specify to the compiler that that section is not allowed to be short?

[H.Becker]

Can you give some more details? Analyze/object of CD_COMMON$START, CD_COMMON and CD_COMMON$END (although the last module did not seem to trigger a linker warning) would do, the default output with the section header table. Or, as the command has /map/full, (an extract of) the map showing these modules and their sections: the 'Program Section Synopsis'.

What is the purpose of CD_COMMON$START? It looks like a "marker" to have a variable/address of the begin of the common area.  Are there also markers  in CD_COMMON and none in CD_COMMON$END?

[stephenbrayshaw]

Start and end both contain values used as markers.

I have attached the output of Analyze/object

[H.Becker]

Module name: "CD_COMMON$END"
Number Type Name File Address File Size Flags
...
5. NOBITS CD_COMMON 0000000000000100 0000000000002000 WA----------------------------

This section is empty and describes uninitialized data, the size is not small. Looks OK.

    Module name:                                "CD_COMMON$START"

Number  Type                 Name                              File Address     File Size         Flags
...
    5.  NOBITS               CD_COMMON                         0000000000000100 0000000000000004  WA------Sho-------------------

This section is empty and describes  uninitialized data, the size is small, so it is a short data section. Looks OK.

    Module name:                                "CD_COMMON"

Number  Type                 Name                              File Address     File Size         Flags
...
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000004916AC0  WA------Sho-------------------

This section is empty and  describes uninitialized data, the size is big, it should not be in a short data section. Looks WRONG.

The short data segment, or segments when the linker produces two is/are limited to a total of 4MB. The above section is way too big. I would expect the linker to issue a

%ILINK-E-SDATAEXC, size of combined read and read/write short data segment exceeded

You probably don't want the CD_COMMON in CD_COMMON$START to be short, you probably want it placed before CD_COMMON from CD_COMMON (with one in short data and the other not, this will not happen). It is expected that the compiler sets short for CD_COMMON in CD_COMMON$START as the size is small enough. I don't know how in Pascal one would force this out of short data. In C I can use a extern_model like common_block or strict_refdef which then places the data into a "normal" data section.

You also probably don't want the CD_COMMON in CD_COMMON to be short. I have no idea why the Pascal compiler places this in short data. Can you provide the source code, that is the definition of the data? Or, what is different in the data definitions in CD_COMMON$END (normal data section) and CD_COMMON (short data section)?

On the other hand, there is no code in any of these modules. Although not recommended, and only useful as a workaround, you can patch the object modules and clear the SHORT bit for these sections (as mentioned before, you can not have a linker PSECT option with NOSHORT).

[stephenbrayshaw]

CD_COMMON$START contains a single integer value.

CD_COMMON$END contains a packed array of bytes (length over 8000).

 CD_COMMON contains a set of values, a few boolean values and a set of arrays.

Each array is of a different record type which contains various types, boolenas, integers, reals and packed chars amongst others.

One of these arrays has 400,000 records in it alone. 

[H.Becker]

That's what I can almost guess from the analyze/object output, it's not the source code. What I want to see is, whether there are Pascal attributes like GLOBAL, VOLATILE, PSECT, etc. assigned to variables, for all variables in all three modules and in which order within the modules for which data type.  If you don't want/can share the real source code/real names, create/use some generic names. As I said I don't have access to a I64 system with Pascal so I can't try to reproduce what I think you have and what I guess you want.

[Hoff]

Is there any that chance that you can get rid of this COMMON-based position-dependent design, and replace it with private or global sections?  COMMONs have always been, are now, and will be, a bag of hurt.   Map the whole thing with a backing file, if that's necessary.  Otherwise, this looks like you might have a bug in Pascal and that's going to involve the support center.

The Pascal equivalent of the C extern model is Alpha-specific, per the doc.

[stephenbrayshaw · Accepted solution]

Thank you to everyone who has helped with this issue especially H.Becker for his identification of CD_COMMON$END as being in the normal section.

The fix for us was to move everything into COMMON and numbered the variables so they remained in the order we required.

We also created a PSECT with a packed byte array to line up the section to a page boundary.

If you do not line it on a page boundary then crmpsc syscall will fail with an invalid argument error

Examples below:

MEMORY_PAGE_TYPE  = PACKED ARRAY [1 .. MEMORY_PAGE_LEN] OF BYTE;

** Start file **

SharedAreaStartAlign : [GLOBAL, VOLATILE, PSECT(CD_COMMON)] MEMORY_PAGE_TYPE;
SharedAreaStart : [GLOBAL, VOLATILE, COMMON(CD_COMMON_000)] INTEGER;

** CD_COMMON file **

CheckData : [GLOBAL, VOLATILE, COMMON(CD_COMMON_001)] CHECK_DATA_TYPE;

GlobalCreateTime : [GLOBAL, VOLATILE, COMMON(CD_COMMON_002)] DAI_TIME_TYPE;

** End file **

SharedAreaEnd   : [GLOBAL, VOLATILE, COMMON(CD_COMMON_999)] MEMORY_PAGE_TYPE;

[H.Becker]

Just to wrap up what I found when I had access to an I64 with Pascal installed. If you have source modules like

$ type %.pas
 
DISK$USER:[USER]A.PAS;1
 
module a;
var
  start : [global,psect(cd_common)] integer;
end.
 
DISK$USER:[USER]B.PAS;1
 
module b;
var
  whatever : [global,psect(cd_common)] integer;
  data : [global,psect(cd_common)] array [1..2048] of integer;
end.

 
DISK$USER:[USER]C.PAS;1
 
module c;
var
  last : [global,psect(cd_common)] array [1..2000] of integer;
end.
$ 

$ pas a
$ pas b
$ pas c

You end up with short data sections in a.obj and b.obj:

$ pipe analyze/obj %.obj |search sys$pipe common
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000000000004  WA------Sho-------------------
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000000002008  WA------Sho-------------------
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000000001F40  WA----------------------------
$

It seems like Pascal determines whether the data should be in a short data section based on the first variable in a module with attributes, specifiying the PSECT name.

I didn't find any other attribute to force data out of short data and there seems to be no directive or qualifier to avoid short data. Given the SOLITARY option for the linker, some code may expect the first data (here variable start) being on a page boundary. So what can be done is to make the first variables bigger than "short" - prefix with unused data in module a and change the sequence in module b:

$ ty a.pas
module a;
var
  unused : [global,psect(cd_common)] array [1..8192] of char;
  start : [global,psect(cd_common)] integer;
end.
$ ty b.pas
module b;
var
  data : [global,psect(cd_common)] array [1..2048] of integer;
  whatever : [global,psect(cd_common)] integer;
end.

$ pas a
$ pas b
$ pipe analyze/obj %.obj |search sys$pipe common
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000000002008  WA----------------------------
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000000002008  WA----------------------------
    5.  NOBITS               CD_COMMON                         00000000000000F0 0000000000001F40  WA----------------------------
$ 

[John Gillings]

I'd have hoped that multiple modules mapping to the same common data would have identical definitions of that data, which would presumably mean that the PSECT attributes would be the same? 

To have anything other than identical definitions (and preferably a single source of text, using whatever langauge constructs are available), is setting yourself up for errors somewhere sometime. I present this post as evidence!

I'd have further hoped that in 21st century, we'd be using better, safer data sharing mechanisms than overlaid common blocks. That you have to fiddle around with dummy variables to align things in the right place should be ringing alarm bells that it's not a good way to achieve your objectives.

This is a good illustration of the general rule "never lie to a compiler". When a compiler gets conflicting descriptions of the same object, it shouldn't be surprising that it generates conflicting implementations of that object. 

[H.Becker]

> I'd have hoped that multiple modules mapping to the same common data would have identical definitions of that data, which would presumably mean that the PSECT attributes would be the same? 

Common data? There is no COMMON in my example other than the misleading name CD_COMMON. A COMMON is different and has different attributes, like:

$ ty x.pas
module x;
var
  start : [global,common(psect)] integer;
end.
$ 

Yeah, just for fun I gave the COMMON the misleading name PSECT.

$ pas x
$ pipe analyze/object x |search sys$pipe psect
    6.  NOBITS               PSECT                             00000000000000F0 
0000000000000004  WA------------GblOvr----------
$ 

And, as you  can see, this will not go into short data and the section has the PSECT attribute OVR.

> I'd have further hoped that in 21st century, we'd be using better, safer data sharing mechanisms than overlaid common blocks.

The OP said this is a port from Alpha, which has the 21 in the processor model numbers but not automagically in the SW designs :-) Who knows, maybe the SW was written for VAX. And who knows what the reasons were to port this code to I64 - but very likely there was some hope that the port would be easy and would not require a re-design or re-write.

> This is a good illustration of the general rule "never lie to a compiler".

Probably true, but I can't see that the rule was violated, here.

That said and not knowing why this code was written that way, no question, I would have written it differently - or maybe would have lied differently :-)

[stephenbrayshaw]

Thanks once again H.Becker for your continued help.

We have now changed it all back to how it was and then changed the start value in CD_COMMON$START from an integer to a packed array oy bytes that is exactly a page long.

This has then put all of the variables into a normal section and with much less work to change the code.

To give some idea as to why this mechanism is still used in the 12st century it is because the software was originally for vax before moving to microvax and then alpha.

During each migration we make a few changes as possible for stability reasons.

We do not ever force our customers to upgrade to the latest version of our software and due to the industry we work in it is VERY important that we do not do that.

Our new systems work differently than these old systems but are only available for Linux / AIX

[John Reagan]

Gee, I don't look here for a week and along comes a Pascal question.  Nobody thought to email me?

Yes, Harmut figured it out.  The first variable placed in the PSECT determines the 'shortness'.  That is a bug which I thought I described in the release notes about the time I transitioned off of the VMS compilers.   Looking back, we should have extended the PSECT attribute to make the shortness explicit.

Adding something to the front that is larger than 8 bytes should do the trick.  I can fix it on the VSI compiler side.  I can't speak for HP.

I'll also update the release notes.