Operating System - OpenVMS
1748053 Members
4642 Online
108758 Solutions
New Discussion юеВ

Can be r1 (gp) register overwritten by error in code? (run on Itanium with OpenVMS)

 
H.Becker
Honored Contributor

Re: Can be r1 (gp) register overwritten by error in code? (run on Itanium with OpenVMS)

>>>
On HP-UX, when (possibly) transferring from one load module to another, the
PC and GP are fetched from the Procedure Linkage Table and of this is
overwritten, bad things can happen. But typically both are corrupted so you
end up in an invalid location, not the right location but wrong GP.
<<<
Later info states that the shown functions are in one module. On VMS this is usually a source or object module, which is linked into one image. On VMS, when transferring from one image to another image PC and GP are fetched from function descriptors in short data. By default function descriptors live in read-only memory. Only explicitly linking with /segment=short=write makes them (and other linker generated short data) writable.
Dennis Handly
Acclaimed Contributor

Re: Can be r1 (gp) register overwritten by error in code? (run on Itanium with OpenVMS)

>you can see that GP was fine when queue_ev function was called.

On HP-UX GP isn't part of the unwind info so it can't be restored. While doing exception handling, the unwinder can look search the module table by PC and then get a value. Perhaps the debugger is displaying what it should be and for the top frame what's actually there?

>Calling function looks very simple:

Yes, very simple.

> 21058: br.call.sptk.many b0 = 0000030 ;;

You might want to display the value of GP before the call and single step at the instruction level to watch how GP changes.

Is it actually going to 0x30, is this a broken disassembler that doesn't show the actual target, just the offset?

>H.Becker: By default function descriptors live in read-only memory.

On HP-UX this can't occur since these can be dynamically changed at loadtime and the dynamic loader needs to modify them.
H.Becker
Honored Contributor

Re: Can be r1 (gp) register overwritten by error in code? (run on Itanium with OpenVMS)

>>>
On HP-UX this can't occur since these can be dynamically changed at loadtime and the dynamic loader needs to modify them.
<<<
On VMS, the corresponding changes (fixups and relocations) are done in inner mode. Before tranferring control to the program, the pages are set to read-only.

>>>
On HP-UX GP isn't part of the unwind info so it can't be restored. While doing exception handling, the unwinder can look search the module table by PC and then get a value. Perhaps the debugger is displaying what it should be and for the top frame what's actually there?
<<<
This isn't different on VMS, the GP is not in the unwind info. I don't know what the debugger shows, but it seems to behave as you describe.

It looks like it uses r1 for the current call stack and the (image) GP for the other call
stacks in this image. A simple example where I zeroed r1 before calling other functions supports this assumption: SHOW STACK only shows the zero for the current, top call stack,

So the GP of the caller looks good in the SHOW STACK output, but may already be zero for the caller.
Dennis Handly
Acclaimed Contributor

Re: Can be r1 (gp) register overwritten by error in code? (run on Itanium with OpenVMS)

>H.Becker: Before transferring control to the program, the pages are set to read-only.

Ah, we have an option +protect to do that. It would have to align on page boundaries to do mprotect(2), so it isn't done by default.

>So the GP of the caller looks good in the SHOW STACK output, but may already be zero for the caller.

So the debugger is showing you what you want to see but not what it is. :-)
So looking at the caller and the caller's caller may be helpful.
(gdb's info shared will show the GPs and addresses for each shlib.)