Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

Quadword relocations - what are they?

 
SOLVED
Go to solution
John McL
Trusted Contributor

Quadword relocations - what are they?

I see from ANALYZE/IMAGE that the code I am working with has thousands of "quadword relocations" in the Image Activator Fixup Section.

Do these cause overheads at image activation and if so, what cauuses them and how can they be minimised?

It might help if I tell you that the code is a mixture of Cobol and C modules.
36 REPLIES 36
Wim Van den Wyngaert
Honored Contributor

Re: Quadword relocations - what are they?

Volker Halle
Honored Contributor

Re: Quadword relocations - what are they?

John,

please disregard the previous answer !

You will find the description of Linker 'Longword and Quadword Address Fixups' in the OpenVMS Internals and Data Structure Manual Chapter 28.4.4:

A longword address record or quadword address record (or both) exists for each shareable image that is the target of .ADDRESS or .ASCID directives.
...

You will also find some references in the OpenVMS Linker manual.

Volker.
Volker Halle
Honored Contributor

Re: Quadword relocations - what are they?

John,

here is a more complete answer. Note that I'm just the messenger ;-)

Volker.

This has nothing to do with unaligned data. This has nothing to do with performance, once the image is activated.

Which platform? I conclude Alpha because the term "Image Activator Fixup Section" was used. How often is the image activated?

On Alpha you have either a shareable image or a main image linked with /section_binding. (On I64 all images have relocations.)

On I64 and on Alpha, an image is only relocated if it can not be placed at the linker suggested addresses: shareable images and main images installed /resident. In any other case none of the image relocations is applied.

If relocations are applied at image activation it takes time to do them.
This is usually faster on I64 than on Alpha, as a result on how the relocations are retrieved from the image file.

You can not avoid applying them. (You can not assign the to be used addresses in the linker: there are no based shareable images on Alpha or on I64.) But you can try to install your image with shared address data.
With shared address data it is tried to apply all the relocations at installation time. It depends on your application, if it is possible.
That is, if your image can be installed with shared address data. If there is a relocation to be applied to a section, which can not be mapped at installation time (for example a writable process section), using shared address data is turned off.

You can't do much at programming level. Image relocations are necessary if you need an address value. It might help if you can use pointers and offsets instead of always pointers.

If the image is activated only a few times, it isn't worth to do anything.

Wim Van den Wyngaert
Honored Contributor

Re: Quadword relocations - what are they?

A better one :
http://h71000.www7.hp.com/doc/82final/vax-to-itanium-porting.pdf

(look for e.g. quad)

Wim
Wim
Volker Halle
Honored Contributor

Re: Quadword relocations - what are they?

re: Wim,

this has nothing to do with unaligned data !

Please re-read my previous posting.

Volker.
Wim Van den Wyngaert
Honored Contributor

Re: Quadword relocations - what are they?

Please zero me. I tried and failed.

Wim
Wim
John Reagan
Respected Contributor

Re: Quadword relocations - what are they?

Mostly coming from your COBOL code I'll guess.

The compiler generates various data structures that the RTL uses at run-time. Many of these data structures contain addresses of your data. Things like the COBOL CANCEL statement wants to return the program to its initial state. We do that by having the RTL reset all your variables based on the magic structures passed.

Looking back, those structures probably should have been some base address and a bunch of offsets, but I was't involved in those decisions.

There isn't much you can do to prevent COBOL from creating them. The compiler *could* be changed, but that is pretty low on the list given those relocations are a one-time event. Plus the RTL would have to handle older .EXE files and new .EXE files. Gets even nastier trying to mix old and new .OBJ files together.
Hoff
Honored Contributor

Re: Quadword relocations - what are they?

Around cases of system and (in this case) application performance investigations, I'd suggest a more systematic approach.

In this case, toss something like the DECset PCA tool or an analog at this application, and find out where the application is actually spending its time.

And as for the fixups, that depends on how many image activations you do per unit time, and whether that's a central cost. Image activations are comparatively expensive on OpenVMS, though there are techniques (installations, reduced numbers of image activations, splitting up big images into pools of servers and into lighter-weight UI tools, etc) that can be used to reduce the overhead.

John McL
Trusted Contributor

Re: Quadword relocations - what are they?

These responses have indicated that more detail is required in some areas.

The applications are on Alpha. The number of executions of all images that might be impacted runs into the thousands per day. The executables are all Cobol and the shareable images they access might be in Cobol or in C. As I said, many of these images have thousands of quadword relocations so all up there's a lot of image activations going on.

I have discovered that the Cobol compiler generates a lot of .ADDRESS instructions and the main culprit seems to be the LINKAGES section. (I was surprised that the pass by reference system didn't generate machine code like Fortran does.)

I also suspect that quadword relocations are created when moving to a subprogram (i.e. with a PERFORM statement). I've also found that even when I work totally in C some quadword relocations are created when I call functions external to my code (e.g. access the various standard run-time libraries).

These quadword relocations seem to be a performance hit. No response has indicated exactly what's involved with resolving them. Is just one PSECT, $LINK$ accessed or must other PSECTs such as $CODE$ be read and resolved? How fast can Alphas process these? Are we talking 1000/second or 10 times that number? In short, what kind of hit are these executables taking? (Which of course will dictate if it's worth making an effort to improve the situation.)

Assuming the performance hit is worth addressing, what options do I have?

It was suggested to me that having the COBOL program specify the data as EXTERNAL might reduce the number of relocations. Any thoughts?

Are any COBOL compiler qualifiers likely to help? (None seem obvious candidates.)

Would the introduction of global sections for a lot of this data be any help?

Would INSTALL / RESIDENT images help? I think there's too many images to do that for all but it may be feasible to install some of them.

I thank those responding so far but I think the responses haven't advanced things very far.

John McL
Trusted Contributor

Re: Quadword relocations - what are they?

Just briefly ... I've just discovered that the LINKAGE section in Cobol is to obtain the addresses of parameters passed into the current code from a calling program (i.e. not for the current code to make its own calls).

By making this clear maybe I'll save respondents going down certain useless paths.
Phil.Howell
Honored Contributor

Re: Quadword relocations - what are they?

There may be some overhead, but IMHO this will be insignificant in the context
of overall system operation.
see this section in the linker manual
http://h71000.www7.hp.com/doc/83final/4548/4548pro_021.html#index_x_559

The link option /shared=address_data may help but I have never used this
option.

You should (depending on available memory) make sure all shared images are
installed, and the more frequently used executable images.
John Reagan
Respected Contributor

Re: Quadword relocations - what are they?



For C programs calling external routines (actually ANY program calling external routines), the compiler and linker don't know where LIB$GET_VM will eventually reside. So the compiler allocates a quadword in the $LINKAGE$ section and asks the linker to put the address of LIB$GET_VM there. Of course, the linker doesn't know either so it leaves relocation info for the image activator. You have these on all systems (do man ld and man loader on your Unix/Linux system).

For COBOL... COBOL likes address constants, no wait, thats wrong. COBOL LOVES address constants. COBOL is addicted to address constants!

Besides the data structures to re-initialized variables for CANCEL, there are two other big areas.

- For regular initialization, the compiler passes the start/end addresses to a DCOB$ initialization routine (or LIBOTS routine). Since the compiler doesn't know the final address of your data, it allocates quadwords in the $LINKAGE$ section and asks the linker to compute the final address. For regular images, the linker can figure it out. For shareable images, even it doesn't know. It leaves relocations for the image activator.

- For PERFORM, COBOL paragraphs don't know if they should "return" or "fall through". Consider:

PERFORM A.
PERFORM A THRU B.

On the first call to A, it needs to "go back". On the second call, it needs to "fall through" to B. How does it know? You guessed it. Address constants!

At each PERFORM, the compiler shoves the address of the "last" paragraph into a special compiler-generated location. Each paragraph checks this special location against its own address to see where to go next. The result, lots of address constants for code labels.

As mentioned earlier, these relocations are only done once at image activation time. If the program stays running for any length of time, the overhead isn't large. If you run over and over again, it can be noticable.

You might want to install with /OPEN /HEADER and /SHARED. You might also want /RESIDENT (which requires linking with /SECTION_BINDING). A larger XFC cache can't hurt either.
John Reagan
Respected Contributor

Re: Quadword relocations - what are they?

Oh, you asked if there are any COBOL qualifiers to help. Nope. Sorry.

We did so some work on I64 to reduce the number of address constants since there is a relatively small limit in the short-data section. However, on Alpha, there is no actual limit, just your gag reflex on the size and overhead for the address constant relocations. We didn't take that support back to Alpha (it was platform-specific inside the code-generator).

COBOL in shareable images is just painful with respect to address constant relocations.
John McL
Trusted Contributor

Re: Quadword relocations - what are they?

John R,

I've spent the last few hours discovering what you've just told me. Either Grace Hopper or some Digital people have a lot to answer for ;-)

Many of these images will be installed /OPEN/HEADER/SHARED but what does that really save apart from the disk I/O and the potential ability to share the data (if we want that)? I also can't see exactly what advantage /RESIDENT gives apart from what seems to be a contiguous chunk of virtual memory.

Is $IMGFIX run across the images when they are installed in order to resolve as many references as possible? That's the kind of thing we need.

I'd rather not have to break the news that we need to radically redesign of our business processes and code so that more work is done per image activation.
Hoff
Honored Contributor

Re: Quadword relocations - what are they?

INSTALL is the user-accessible portion of the $imgact image activation service; INSTALL pre-activates the image.

Once the image activation processing starts in earnest, the $imgfix then occurs, since the fix-ups aren't known until the images are mapped into the process address space. AFAIK, there's no way to perform the fix-ups until after the executable image and all of the constituent (installed or otherwise) shareable images are all stacked into process virtual address space, and that doesn't happen during the INSTALL processing.

http://h71000.www7.hp.com/doc/83final/4548/4548pro_020.html#fixup_sec

John Gillings
Honored Contributor
Solution

Re: Quadword relocations - what are they?

John,

>installed /OPEN/HEADER/SHARED but what
>does that really save apart from the disk
>I/O and

Sharing writeable data is a special case, but sharing code and readonly data can be a big win if you have many processes executing the same images simultaneously. You save on memory, because there's only one copy, and pagefaults because a page you want may be in the working set of another process, so you get a global valid fault rather than a hard fault.

> I also can't see exactly what
>advantage /RESIDENT gives apart from what
>seems to be a contiguous chunk of virtual
>memory.

The code is physically resident, so it's paged in at INSTALL time. No pagefaults required for references. It's also mapped into S0/S1 space, so the addresses are the same for all processes and fixed at installation time. That means you it's not necessary to perform relocations. The cost is physical memory, which is getting cheaper and cheaper.

Also look at /SHARED=ADDRESS_DATA. From HELP:

"When you use the ADDRESS_DATA keyword with the /SHARED qualifier, P1 space addresses are assigned for shareable images. With the assigned addresses, the Install utility can determine the content of an address data section when the image is installed rather than when it is activated, reducing CPU and I/O time. A global section is created to allow shared access to address data image sections."

All this presupposes that the time to perform the fixups and the number of image activations are significant enough to worry about.

Try writing a test program that just activates your shareable images then exits immediately. Compare the execution times with different combinations of installed images and different options.

Depending on what the images do, it may be better to try to keep the image running longer (for example, put a menu or event loop into the image, rather than in a DCL jacket). As Hoff suggests, another approach is to go "client server". Have the heavy weight processing in a long lived server process, with a lightweight client feeding it requests and catching results. The (lower) cost of starting and stopping the client may compensate for the cost of interprocess communications.
A crucible of informative mistakes
John McL
Trusted Contributor

Re: Quadword relocations - what are they?

To Hoff (Feb 26, 01:06),

I realised what I was asking after I had posted. Life would be a lot easier if, as John R said, Cobol used offsets. Even the literals are set up with .ADDRESS and can't be resolved until activation.


To John G,

I've used installed images many times in the last 29 years :-) but I haven't worked with /RESIDENT images. I understood that the only way to replace them was with a reboot but that's not an option here because changes are frequently made to the shareable images and they need to be replaced.

The use of /SHARED_ADDRESS looks potentially useful but I wonder how well it works with COBOL. It's something I'll try today or tomorrow, likewise a test program that accesses these COBOL shareable images and then exits (to determine what the activation overhead really is).

Your other suggestions refer to a major redesign. We'd prefer not to go that way but it looks like we may not have a choice.
Phil.Howell
Honored Contributor

Re: Quadword relocations - what are they?

What leads you to think that the image activation fixups are causing a performance problem?
Do you have lots (more than 100) of symbol_vectors in the link options for your sharable images?
Phil

To JR
on the other side of the world we usually stick the reply text in our paste buffer for when itrc fails (again) I thought you might get better response locally.
John McL
Trusted Contributor

Re: Quadword relocations - what are they?

Phill,

When the number of quadword relocations is over 5000 I think there's a good chance that activation will be time-consuming, and we have plenty of such images. By the way, that figure comes from the shareable image so any image that references it will also have its own quadword relocations to be resolved.

We have quite a number of Symbol Vectors defined for the Link of the shareable image but I don't believe it would above 50 in any one shareable. Would that really make a difference when the activation can't distinguish the entry points that we might access from those that we don't?

John Gillings
Honored Contributor

Re: Quadword relocations - what are they?

John (McL)

>I understood that the only way to replace
>them was with a reboot but that's not an
>option

Not quite true. Obviously you can't yank resident memory out from underneath a running process. That means for some very widely used shareable images (for example, DECC$SHR - potentially referenced by processes that last forever), if you want to replace the shareable image, you either need to have enough resident memory to keep two or more copies of the image, or you need to reboot to force long running processes to relinquish the superceeded memory.

If you're in control of all the processes that activate the resident image, there's no reason you can't replace it without having to reboot. It can get a bit tricky to ensure you've got everyone out, AND, if your changes make the image bigger, to avoid checkerboarding your resident memory you'll need to over allocate a fairly significant amount. That means the "spare" memory is effectively wasted because the system can't use it for any other purpose.

Unless you're very careful, you can get yourself into a position where an image has mostly been removed, but it's not quite complete, so you can't activate it, and you can't replace it - time to reboot! :-(

Unfortunately INSTALL hasn't been written for frequent REPLACEs and there a few leaks and bugs lurking if you exercise it too vigourously.

John R, as Phil says, over this side of the planet, we EXPECT at least 2-3 failures attempting to post anything to ITRC. I always COPY my response before hitting SUBMIT. (finest technology on the planet, but do they use it properly?...)
A crucible of informative mistakes
Hoff
Honored Contributor

Re: Quadword relocations - what are they?

5000 additions (base + offset) and some related data structure traversal baggage might well be less costly in terms of aggregate performance than a single disk I/O.

I'd look to measure the performance; not assume it. I'm skeptical about assumptions around performance and tuning in general, having encountered enough unexpected limits over the years. Using tools such as PCA really help. Certainly, do try a constructed image with 5000 and 50000 fix-ups here. But don't get tunneled into one particular aspect.

Do determine if the box you're working with is worth dealing with, for instance, and whether the box has sufficient free memory and sufficient quotas and working set settings.

--

ITRC simply isn't stable, and fully appears to be operating well beyond its available hardware or outside its intended operational envelope.
x2084
Trusted Contributor

Re: Quadword relocations - what are they?

> Would INSTALL / RESIDENT images help? I
> think there's too many images to
> that for all but it may be feasible to
> install some of them.

Maybe what was already said was not clear, maybe it should be just repeated:

You can not avoid applying them (image relocations). ... But you can try to install your image with shared address data. With shared address data it is tried to apply all the relocations at installation time. It depends on your application, if it is possible. ... If there is a relocation to be applied to a section, which can not be mapped at installation time (for example a writable process section), using shared address data is turned off.

Installing /RESIDENT makes code (and/or data) resident for running the code and accessing the data. Here we talk about applying relocations which takes place before any application code is running. /RESIDENT alone does not help.

What you want to try is INSTALL ADD/SHARE=ADDRESS. Without the =address no
relocation is applied at installation time: the image is not pre-activated. With =address fixups ARE also applied. That's the whole purpose of installing a shareable image with shared address data. The whole image (including the address data) is mapped into P1 at a fixed location for all processes. That makes it possible to pre-apply anything.

That requires to install ALL the shareable images with shared address data, which your image(s) depend on. Otherwise this can not work.

But as said above, you may not be able to install your image with shared address data for other reasons.

At least for a test, you might want to link your main image with object modules and not with shareable images. That should reduce both, relocations and fixups. But we are not talking about fixups (which are the references to other shareable
images), or?

> We have quite a number of Symbol Vectors
> defined for the Link of the shareable
> image but I don't believe it would above 50
> in any one shareable. Would that
> really make a difference when the
> activation can't distinguish the entry
> points that we might access from those that
> we don't?

In the output of ANALYZE/IMAGE the fixups are shown as "Quadword .ADDRESS Reference Fixups" and they all have a header line showing the image from which the fixup is applied. The linker only generates a fixup for those entries which are really used. The (Alpha) linker might generate more than one fixup for
the same entry. Usually it doesn't.

> Unfortunately INSTALL hasn't been written
> for frequent REPLACEs and there a
> few leaks and bugs lurking if you exercise
> it too vigourously.

True. But, which version of VMS? A lot of bugs and leaks in this area are fixed
in 8.3 (both I64 and Alpha).

Hartmut Becker
Robert Gezelter
Honored Contributor

Re: Quadword relocations - what are they?

John,

I will agree with what has been said about the scale of the overhead.

Once upon a time, when machines were orders of magnitude slower, it was perhaps a concern. 5000 relocations on a 1 gip (10**9 instuctions/second) processor, less than (as was observed) the overhead of a single disk IO.

I have used shareable images extensively for many years, and despite having used large numbers of shareable images, the overhead from the extra image activation overhead has never been a significant issue.

Before going into the deep waters of a re-write, I would suggest looking at other performance optimizations that can be achieved without actually touching code. It is far cheaper, even when outside expertise is retained.

- Bob Gezelter, http://www.rlgsc.com
John McL
Trusted Contributor

Re: Quadword relocations - what are they?

To John Gillings,

(Sorry about the delay. Access to ITRC was timing out yesterday.)

The team that I'm a part of does not have control over the processes that might activate a resident image.

I've also just taken a look at one shareable image which might not even be the largest. It has just short of 49,000 quadword relocations (out of interest) and it has 5 PSECTs that each have a size of just over 33 million bytes, as well as several other PSECTs. Let's say it needs 160 MB all up. If I understand your logic correctly we would need to allow about 2.5 x 160MB, i.e. about 400MB, for this one shareable image. (Do I understand you correctly ?)

That doesn't sound terribly feasible to me, especially when we have no control over access to the shareable images.