Quadword relocations - what are they?

[John Gillings]

John (McL)

>I understood that the only way to replace
>them was with a reboot but that's not an
>option

Not quite true. Obviously you can't yank resident memory out from underneath a running process. That means for some very widely used shareable images (for example, DECC$SHR - potentially referenced by processes that last forever), if you want to replace the shareable image, you either need to have enough resident memory to keep two or more copies of the image, or you need to reboot to force long running processes to relinquish the superceeded memory.

If you're in control of all the processes that activate the resident image, there's no reason you can't replace it without having to reboot. It can get a bit tricky to ensure you've got everyone out, AND, if your changes make the image bigger, to avoid checkerboarding your resident memory you'll need to over allocate a fairly significant amount. That means the "spare" memory is effectively wasted because the system can't use it for any other purpose.

Unless you're very careful, you can get yourself into a position where an image has mostly been removed, but it's not quite complete, so you can't activate it, and you can't replace it - time to reboot! :-(

Unfortunately INSTALL hasn't been written for frequent REPLACEs and there a few leaks and bugs lurking if you exercise it too vigourously.

John R, as Phil says, over this side of the planet, we EXPECT at least 2-3 failures attempting to post anything to ITRC. I always COPY my response before hitting SUBMIT. (finest technology on the planet, but do they use it properly?...)

[Hoff]

5000 additions (base + offset) and some related data structure traversal baggage might well be less costly in terms of aggregate performance than a single disk I/O.

I'd look to measure the performance; not assume it. I'm skeptical about assumptions around performance and tuning in general, having encountered enough unexpected limits over the years. Using tools such as PCA really help. Certainly, do try a constructed image with 5000 and 50000 fix-ups here. But don't get tunneled into one particular aspect.

Do determine if the box you're working with is worth dealing with, for instance, and whether the box has sufficient free memory and sufficient quotas and working set settings.

--

ITRC simply isn't stable, and fully appears to be operating well beyond its available hardware or outside its intended operational envelope.

[x2084]

> Would INSTALL / RESIDENT images help? I
> think there's too many images to
> that for all but it may be feasible to
> install some of them.

Maybe what was already said was not clear, maybe it should be just repeated:

You can not avoid applying them (image relocations). ... But you can try to install your image with shared address data. With shared address data it is tried to apply all the relocations at installation time. It depends on your application, if it is possible. ... If there is a relocation to be applied to a section, which can not be mapped at installation time (for example a writable process section), using shared address data is turned off.

Installing /RESIDENT makes code (and/or data) resident for running the code and accessing the data. Here we talk about applying relocations which takes place before any application code is running. /RESIDENT alone does not help.

What you want to try is INSTALL ADD/SHARE=ADDRESS. Without the =address no
relocation is applied at installation time: the image is not pre-activated. With =address fixups ARE also applied. That's the whole purpose of installing a shareable image with shared address data. The whole image (including the address data) is mapped into P1 at a fixed location for all processes. That makes it possible to pre-apply anything.

That requires to install ALL the shareable images with shared address data, which your image(s) depend on. Otherwise this can not work.

But as said above, you may not be able to install your image with shared address data for other reasons.

At least for a test, you might want to link your main image with object modules and not with shareable images. That should reduce both, relocations and fixups. But we are not talking about fixups (which are the references to other shareable
images), or?

> We have quite a number of Symbol Vectors
> defined for the Link of the shareable
> image but I don't believe it would above 50
> in any one shareable. Would that
> really make a difference when the
> activation can't distinguish the entry
> points that we might access from those that
> we don't?

In the output of ANALYZE/IMAGE the fixups are shown as "Quadword .ADDRESS Reference Fixups" and they all have a header line showing the image from which the fixup is applied. The linker only generates a fixup for those entries which are really used. The (Alpha) linker might generate more than one fixup for
the same entry. Usually it doesn't.

> Unfortunately INSTALL hasn't been written
> for frequent REPLACEs and there a
> few leaks and bugs lurking if you exercise
> it too vigourously.

True. But, which version of VMS? A lot of bugs and leaks in this area are fixed
in 8.3 (both I64 and Alpha).

Hartmut Becker

[Robert Gezelter]

John,

I will agree with what has been said about the scale of the overhead.

Once upon a time, when machines were orders of magnitude slower, it was perhaps a concern. 5000 relocations on a 1 gip (10**9 instuctions/second) processor, less than (as was observed) the overhead of a single disk IO.

I have used shareable images extensively for many years, and despite having used large numbers of shareable images, the overhead from the extra image activation overhead has never been a significant issue.

Before going into the deep waters of a re-write, I would suggest looking at other performance optimizations that can be achieved without actually touching code. It is far cheaper, even when outside expertise is retained.

- Bob Gezelter, http://www.rlgsc.com

[John McL]

To John Gillings,

(Sorry about the delay. Access to ITRC was timing out yesterday.)

The team that I'm a part of does not have control over the processes that might activate a resident image.

I've also just taken a look at one shareable image which might not even be the largest. It has just short of 49,000 quadword relocations (out of interest) and it has 5 PSECTs that each have a size of just over 33 million bytes, as well as several other PSECTs. Let's say it needs 160 MB all up. If I understand your logic correctly we would need to allow about 2.5 x 160MB, i.e. about 400MB, for this one shareable image. (Do I understand you correctly ?)

That doesn't sound terribly feasible to me, especially when we have no control over access to the shareable images.

[John McL]

To Hoff,

Today I'll write a test harness and run various options.
- access the shareable image with LIB$FIND_IMAGE_SYMBOL (which is often used here)
- linked shareable image (i.e. normal run-time library)
- move called routine into an object library and link against that (although in practice other shareable images would probably be called as well).

I'll report back here with the findings.

[John McL]

As Harmut mentioned and John Gillings mentioned earlier ... and I forgot to note in the above msg ... I'll also try to get access to experiment with /RESIDENT and /SHARED=ADDRESS. Setting up this test might not be so easy.

And these tests should keep Bob G happy too :-)

[John McL]

I'm still running tests but here's some initial data.

I'm activating a main program which then accesses a shareable image with a variable number of functions (and hence a variable number of quadword relocations) ...

For 5000 image activations

QR's = 7002 time = 57 seconds
QR's = 150894 time = 109 seconds
QR's = 712279 time = 329 seconds

In this case the shareable image was not installed but both it and the main image would have been cached.

I'm starting to think that I need to look for a balance between functions in run-time libraries and linking QR-heavy functions directly into the programs that use them (i.e. don't hurt every image that accesses the RTL but doesn't use those QR-heavy functions).

I should have more information in about 18 hours.

[John Gillings]

John,

> look for a balance between functions in
>run-time libraries and linking QR-heavy
>functions directly into the programs that
>use them

That's a bit of a rod for your own back. If you change a routine, you need to locate all references and relink them. Then you've got all the potential nasties associated with mismatched code incompatibilities.

Depending on how the routines are used, you may be able to reduce the run time impact by activating images only when actually needed. One way to do this automatically would be to create a "lightweight" shareable image that effectively only defines the entry points. This should improve your activation time to the first executable statement.

When the a routine is first called it activates the real image (using LIB$FIND_IMAGE_SYMBOL) and revectors calls through. My FAKE_RTL.COM (See OpenVMS Technical Journal) is a mechanism that will produce the lightweight shareable image automatically (though you may want to cut out the argument tracing code). Overall you're using more resources (potentially doubling the actual number of shareable images) and do more work because each call gets turned into two, but you may find it's subjectively faster because the cost of activating a particular image is only incurred when and if it's required.

[John Gillings]

Just occurred to me...

> (i.e. don't hurt every image that
>accesses the RTL but doesn't use those QR-
>heavy functions).

Another use for the FAKE_RTL technique here would be to gather statistics on actual use of different routines. You might be able to use that information to redistribute routines across different shareable images to give a better balance.

Isolate the expensive, rarely used ones in their own shareable image, possibly to be activated on demand. Put the cheap freqently used ones in a different image to be activated directly.

Your best weapon in improving performance is good quality data.