topic Re: Quadword relocations - what are they? in Operating System - OpenVMS

Quadword relocations - what are they?

John McL — Mon, 25 Feb 2008 03:59:30 GMT

I see from ANALYZE/IMAGE that the code I am working with has thousands of "quadword relocations" in the Image Activator Fixup Section.

Do these cause overheads at image activation and if so, what cauuses them and how can they be minimised?

It might help if I tell you that the code is a mixture of Cobol and C modules.

Re: Quadword relocations - what are they?

Wim Van den Wyngaert — Mon, 25 Feb 2008 08:23:58 GMT

http://h71000.www7.hp.com/doc/82final/5601/aa-pv64e-te.pdf

Chapter 4.

Wim

Re: Quadword relocations - what are they?

Volker Halle — Mon, 25 Feb 2008 09:20:09 GMT

John,

please disregard the previous answer !

You will find the description of Linker 'Longword and Quadword Address Fixups' in the OpenVMS Internals and Data Structure Manual Chapter 28.4.4:

A longword address record or quadword address record (or both) exists for each shareable image that is the target of .ADDRESS or .ASCID directives.
...

You will also find some references in the OpenVMS Linker manual.

Volker.

Re: Quadword relocations - what are they?

Volker Halle — Mon, 25 Feb 2008 09:55:35 GMT

John,

here is a more complete answer. Note that I'm just the messenger ;-)

Volker.

This has nothing to do with unaligned data. This has nothing to do with performance, once the image is activated.

Which platform? I conclude Alpha because the term "Image Activator Fixup Section" was used. How often is the image activated?

On Alpha you have either a shareable image or a main image linked with /section_binding. (On I64 all images have relocations.)

On I64 and on Alpha, an image is only relocated if it can not be placed at the linker suggested addresses: shareable images and main images installed /resident. In any other case none of the image relocations is applied.

If relocations are applied at image activation it takes time to do them.
This is usually faster on I64 than on Alpha, as a result on how the relocations are retrieved from the image file.

You can not avoid applying them. (You can not assign the to be used addresses in the linker: there are no based shareable images on Alpha or on I64.) But you can try to install your image with shared address data.
With shared address data it is tried to apply all the relocations at installation time. It depends on your application, if it is possible.
That is, if your image can be installed with shared address data. If there is a relocation to be applied to a section, which can not be mapped at installation time (for example a writable process section), using shared address data is turned off.

You can't do much at programming level. Image relocations are necessary if you need an address value. It might help if you can use pointers and offsets instead of always pointers.

If the image is activated only a few times, it isn't worth to do anything.

Re: Quadword relocations - what are they?

Wim Van den Wyngaert — Mon, 25 Feb 2008 11:00:00 GMT

A better one :
http://h71000.www7.hp.com/doc/82final/vax-to-itanium-porting.pdf

(look for e.g. quad)

Wim

Re: Quadword relocations - what are they?

Volker Halle — Mon, 25 Feb 2008 11:18:07 GMT

re: Wim,

this has nothing to do with unaligned data !

Please re-read my previous posting.

Volker.

Re: Quadword relocations - what are they?

Wim Van den Wyngaert — Mon, 25 Feb 2008 11:55:00 GMT

Please zero me. I tried and failed.

Wim

Re: Quadword relocations - what are they?

John Reagan — Mon, 25 Feb 2008 18:53:55 GMT

Mostly coming from your COBOL code I'll guess.

The compiler generates various data structures that the RTL uses at run-time. Many of these data structures contain addresses of your data. Things like the COBOL CANCEL statement wants to return the program to its initial state. We do that by having the RTL reset all your variables based on the magic structures passed.

Looking back, those structures probably should have been some base address and a bunch of offsets, but I was't involved in those decisions.

There isn't much you can do to prevent COBOL from creating them. The compiler *could* be changed, but that is pretty low on the list given those relocations are a one-time event. Plus the RTL would have to handle older .EXE files and new .EXE files. Gets even nastier trying to mix old and new .OBJ files together.

Re: Quadword relocations - what are they?

Hoff — Mon, 25 Feb 2008 19:58:19 GMT

Around cases of system and (in this case) application performance investigations, I'd suggest a more systematic approach.

In this case, toss something like the DECset PCA tool or an analog at this application, and find out where the application is actually spending its time.

And as for the fixups, that depends on how many image activations you do per unit time, and whether that's a central cost. Image activations are comparatively expensive on OpenVMS, though there are techniques (installations, reduced numbers of image activations, splitting up big images into pools of servers and into lighter-weight UI tools, etc) that can be used to reduce the overhead.

Re: Quadword relocations - what are they?

John McL — Mon, 25 Feb 2008 21:46:02 GMT

These responses have indicated that more detail is required in some areas.

The applications are on Alpha. The number of executions of all images that might be impacted runs into the thousands per day. The executables are all Cobol and the shareable images they access might be in Cobol or in C. As I said, many of these images have thousands of quadword relocations so all up there's a lot of image activations going on.

I have discovered that the Cobol compiler generates a lot of .ADDRESS instructions and the main culprit seems to be the LINKAGES section. (I was surprised that the pass by reference system didn't generate machine code like Fortran does.)

I also suspect that quadword relocations are created when moving to a subprogram (i.e. with a PERFORM statement). I've also found that even when I work totally in C some quadword relocations are created when I call functions external to my code (e.g. access the various standard run-time libraries).

These quadword relocations seem to be a performance hit. No response has indicated exactly what's involved with resolving them. Is just one PSECT, $LINK$ accessed or must other PSECTs such as $CODE$ be read and resolved? How fast can Alphas process these? Are we talking 1000/second or 10 times that number? In short, what kind of hit are these executables taking? (Which of course will dictate if it's worth making an effort to improve the situation.)

Assuming the performance hit is worth addressing, what options do I have?

It was suggested to me that having the COBOL program specify the data as EXTERNAL might reduce the number of relocations. Any thoughts?

Are any COBOL compiler qualifiers likely to help? (None seem obvious candidates.)

Would the introduction of global sections for a lot of this data be any help?

Would INSTALL / RESIDENT images help? I think there's too many images to do that for all but it may be feasible to install some of them.

I thank those responding so far but I think the responses haven't advanced things very far.

Re: Quadword relocations - what are they?

John McL — Mon, 25 Feb 2008 22:47:15 GMT

Just briefly ... I've just discovered that the LINKAGE section in Cobol is to obtain the addresses of parameters passed into the current code from a calling program (i.e. not for the current code to make its own calls).

By making this clear maybe I'll save respondents going down certain useless paths.

Re: Quadword relocations - what are they?

Phil.Howell — Mon, 25 Feb 2008 23:30:57 GMT

There may be some overhead, but IMHO this will be insignificant in the context
of overall system operation.
see this section in the linker manual
http://h71000.www7.hp.com/doc/83final/4548/4548pro_021.html#index_x_559

The link option /shared=address_data may help but I have never used this
option.

You should (depending on available memory) make sure all shared images are
installed, and the more frequently used executable images.

Re: Quadword relocations - what are they?

John Reagan — Mon, 25 Feb 2008 23:44:05 GMT

For C programs calling external routines (actually ANY program calling external routines), the compiler and linker don't know where LIB$GET_VM will eventually reside. So the compiler allocates a quadword in the $LINKAGE$ section and asks the linker to put the address of LIB$GET_VM there. Of course, the linker doesn't know either so it leaves relocation info for the image activator. You have these on all systems (do man ld and man loader on your Unix/Linux system).

For COBOL... COBOL likes address constants, no wait, thats wrong. COBOL LOVES address constants. COBOL is addicted to address constants!

Besides the data structures to re-initialized variables for CANCEL, there are two other big areas.

- For regular initialization, the compiler passes the start/end addresses to a DCOB$ initialization routine (or LIBOTS routine). Since the compiler doesn't know the final address of your data, it allocates quadwords in the $LINKAGE$ section and asks the linker to compute the final address. For regular images, the linker can figure it out. For shareable images, even it doesn't know. It leaves relocations for the image activator.

- For PERFORM, COBOL paragraphs don't know if they should "return" or "fall through". Consider:

PERFORM A.
PERFORM A THRU B.

On the first call to A, it needs to "go back". On the second call, it needs to "fall through" to B. How does it know? You guessed it. Address constants!

At each PERFORM, the compiler shoves the address of the "last" paragraph into a special compiler-generated location. Each paragraph checks this special location against its own address to see where to go next. The result, lots of address constants for code labels.

As mentioned earlier, these relocations are only done once at image activation time. If the program stays running for any length of time, the overhead isn't large. If you run over and over again, it can be noticable.

You might want to install with /OPEN /HEADER and /SHARED. You might also want /RESIDENT (which requires linking with /SECTION_BINDING). A larger XFC cache can't hurt either.

Re: Quadword relocations - what are they?

John Reagan — Mon, 25 Feb 2008 23:54:31 GMT

Oh, you asked if there are any COBOL qualifiers to help. Nope. Sorry.

We did so some work on I64 to reduce the number of address constants since there is a relatively small limit in the short-data section. However, on Alpha, there is no actual limit, just your gag reflex on the size and overhead for the address constant relocations. We didn't take that support back to Alpha (it was platform-specific inside the code-generator).

COBOL in shareable images is just painful with respect to address constant relocations.

Re: Quadword relocations - what are they?

John McL — Mon, 25 Feb 2008 23:56:00 GMT

John R,

I've spent the last few hours discovering what you've just told me. Either Grace Hopper or some Digital people have a lot to answer for ;-)

Many of these images will be installed /OPEN/HEADER/SHARED but what does that really save apart from the disk I/O and the potential ability to share the data (if we want that)? I also can't see exactly what advantage /RESIDENT gives apart from what seems to be a contiguous chunk of virtual memory.

Is $IMGFIX run across the images when they are installed in order to resolve as many references as possible? That's the kind of thing we need.

I'd rather not have to break the news that we need to radically redesign of our business processes and code so that more work is done per image activation.

Re: Quadword relocations - what are they?

Hoff — Tue, 26 Feb 2008 01:06:37 GMT

INSTALL is the user-accessible portion of the $imgact image activation service; INSTALL pre-activates the image.

Once the image activation processing starts in earnest, the $imgfix then occurs, since the fix-ups aren't known until the images are mapped into the process address space. AFAIK, there's no way to perform the fix-ups until after the executable image and all of the constituent (installed or otherwise) shareable images are all stacked into process virtual address space, and that doesn't happen during the INSTALL processing.

http://h71000.www7.hp.com/doc/83final/4548/4548pro_020.html#fixup_sec

Re: Quadword relocations - what are they?

John Gillings — Tue, 26 Feb 2008 01:17:20 GMT

John,

>installed /OPEN/HEADER/SHARED but what
>does that really save apart from the disk
>I/O and

Sharing writeable data is a special case, but sharing code and readonly data can be a big win if you have many processes executing the same images simultaneously. You save on memory, because there's only one copy, and pagefaults because a page you want may be in the working set of another process, so you get a global valid fault rather than a hard fault.

> I also can't see exactly what
>advantage /RESIDENT gives apart from what
>seems to be a contiguous chunk of virtual
>memory.

The code is physically resident, so it's paged in at INSTALL time. No pagefaults required for references. It's also mapped into S0/S1 space, so the addresses are the same for all processes and fixed at installation time. That means you it's not necessary to perform relocations. The cost is physical memory, which is getting cheaper and cheaper.

Also look at /SHARED=ADDRESS_DATA. From HELP:

"When you use the ADDRESS_DATA keyword with the /SHARED qualifier, P1 space addresses are assigned for shareable images. With the assigned addresses, the Install utility can determine the content of an address data section when the image is installed rather than when it is activated, reducing CPU and I/O time. A global section is created to allow shared access to address data image sections."

All this presupposes that the time to perform the fixups and the number of image activations are significant enough to worry about.

Try writing a test program that just activates your shareable images then exits immediately. Compare the execution times with different combinations of installed images and different options.

Depending on what the images do, it may be better to try to keep the image running longer (for example, put a menu or event loop into the image, rather than in a DCL jacket). As Hoff suggests, another approach is to go "client server". Have the heavy weight processing in a long lived server process, with a lightweight client feeding it requests and catching results. The (lower) cost of starting and stopping the client may compensate for the cost of interprocess communications.

Re: Quadword relocations - what are they?

John McL — Tue, 26 Feb 2008 01:44:08 GMT

To Hoff (Feb 26, 01:06),

I realised what I was asking after I had posted. Life would be a lot easier if, as John R said, Cobol used offsets. Even the literals are set up with .ADDRESS and can't be resolved until activation.

To John G,

I've used installed images many times in the last 29 years :-) but I haven't worked with /RESIDENT images. I understood that the only way to replace them was with a reboot but that's not an option here because changes are frequently made to the shareable images and they need to be replaced.

The use of /SHARED_ADDRESS looks potentially useful but I wonder how well it works with COBOL. It's something I'll try today or tomorrow, likewise a test program that accesses these COBOL shareable images and then exits (to determine what the activation overhead really is).

Your other suggestions refer to a major redesign. We'd prefer not to go that way but it looks like we may not have a choice.

Re: Quadword relocations - what are they?

Phil.Howell — Tue, 26 Feb 2008 02:08:28 GMT

What leads you to think that the image activation fixups are causing a performance problem?
Do you have lots (more than 100) of symbol_vectors in the link options for your sharable images?
Phil

To JR
on the other side of the world we usually stick the reply text in our paste buffer for when itrc fails (again) I thought you might get better response locally.

Re: Quadword relocations - what are they?

John McL — Tue, 26 Feb 2008 02:29:12 GMT

Phill,

When the number of quadword relocations is over 5000 I think there's a good chance that activation will be time-consuming, and we have plenty of such images. By the way, that figure comes from the shareable image so any image that references it will also have its own quadword relocations to be resolved.

We have quite a number of Symbol Vectors defined for the Link of the shareable image but I don't believe it would above 50 in any one shareable. Would that really make a difference when the activation can't distinguish the entry points that we might access from those that we don't?