library difference between compile env. and production env.

[Dennis Handly]

>Here you go,
>(gdb) disas 0xc80cfb68 0xc80cfb68+4*8

Rats, copied the wrong number. That should be the one in the call:
(gdb) disas 0xc89cfb58 0xc89cfb58+4*8

>If you think that it is link problem,
can you tell me whether or not the following link is OK?
-lrwtool_v2 -lstd_v2 -lCsup_v2 -lc_r -lc

Ack!!

If you don't use tools.h++, you should remove -lrwtool_v2.

Did you see this from?
http://www.docs.hp.com/en/7762/5991-4874/distributing.htm#linking

You should NOT be adding -lc to shlibs!

And the most scary thing is that you have -lc_r, which is for 9.X only!

[tash S]

Hi Dennis,

Thanks,
> You should NOT be adding -lc to shlibs!

Our code is used *.c codes as well as C++ code. So do we need to put -lc as linker ?

[tash S]

Hi Dennis,

Even if we changed the link library to;

-lstd_v2 -lCsup_v2

The core is appeared.

Here is disas result:

=========================
(gdb) disas 0xc818fb18 0xc818fb18+4*8
Dump of assembler code from 0xc818fb18 to 0xc818fb38:
;;; File: /opt/aCC/include_std/rw/tree
;;; 405 _C_link_type _C_get_link() {
0xc818fb18 <__rw::__rb_tree,__r
w::__select1st<:pair>,RSACString *>,rw_deref_compar
e<:less>,RSACString>,std::allocator<:pair>har *>>>::_C_get_link(void)>: stw %rp,-0x14(%sp)
0xc818fb1c: ldo 0x40(%sp),%sp
0xc818fb20: stw %r19,-0x20(%sp)
0xc818fb24: nop
0xc818fb28: stw %r26,-0x64(%sp)
;;; 406 _C_link_type __tmp = _C_free_list;
0xc818fb2c: ldw -0x64(%sp),%r20
0xc818fb30: ldo 4(%r20),%r21
0xc818fb34: ldw 0(%r21),%r22
End of assembler dump.
=========================

[tash S]

My library that has problem is here for ldd command.

bash-2.04# aaa.sl
/usr/lib/libCsup_v2.2 => /usr/lib/libCsup_v2.2
/usr/lib/libstd_v2.2 => /usr/lib/libstd_v2.2
../lib/libapr-0.sl.9 => ../lib/libapr-0.sl.9
/usr/lib/libdld.2 => /usr/lib/libdld.2
/usr/lib/libc.2 => /usr/lib/libc.2
/usr/lib/libdld.2 => /usr/lib/libdld.2
/usr/lib/libnsl.1 => /usr/lib/libnsl.1
/usr/lib/libxti.2 => /usr/lib/libxti.2
/usr/lib/libm.2 => /usr/lib/libm.2
/usr/lib/libc.2 => /usr/lib/libc.2
/usr/lib/libCsup_v2.2 => /usr/lib/libCsup_v2.2
/usr/lib/libstd_v2.2 => /usr/lib/libstd_v2.2
/usr/lib/librwtool_v2.2 => /usr/lib/librwtool_v2.2
/usr/lib/libnsl.1 => /usr/lib/libnsl.1

[Dennis Handly]

>Our code is used *.c codes as well as C++ code. So do we need to put -lc as linker?

No, you should assume the executable is linked with libc.

>Even if we changed the link library to: -lstd_v2 -lCsup_v2
The core is appeared.

Rats.

>Here is disas result:
(gdb) disas 0xc818fb18 0xc818fb18+4*8

Where did you get these addresses??
I said 0xc89cfb58. And if you relinked it, you would have to get the address from a link that looked like:
0xc89cfbdc <::_C_get_link(void)+0x74>:
call 0xc89cfb58 <::_C_add_new_buffer(void)>

> My library that has problem is here for ldd command.
/usr/lib/libCsup_v2.2
/usr/lib/libstd_v2.2
../lib/libapr-0.sl.9
/usr/lib/libdld.2
/usr/lib/libc.2
/usr/lib/libnsl.1
/usr/lib/libxti.2
/usr/lib/libm.2
/usr/lib/librwtool_v2.2

You should also try chatr(1) so we don't see the duplicates.

[tash S]

Here is the result of chatr

bash-2.04# chatr aaa.sl
aaa.sl:
shared library
shared library dynamic path search:
SHLIB_PATH disabled second
embedded path disabled first Not Defined
shared library list:
dynamic ../lib/libapr-0.sl.9
dynamic /usr/lib/libstd_v2.2
dynamic /usr/lib/libCsup_v2.2
shared vtable support disabled
explicit unloading enabled
static branch prediction disabled
executable from stack: D (default)
kernel assisted branch prediction enabled
lazy swap allocation disabled
text segment locking disabled
data segment locking disabled
third quadrant private data space disabled
fourth quadrant private data space disabled
third quadrant global data space disabled
data page size: D (default)
instruction page size: D (default)
nulptr references disabled
shared library private mapping disabled

[Dennis Handly]

So all of those libs comes from libapr-0.sl.9. So can you make changes there too?

Remove -lc and librwtool_v2.2 if not needed.

[tash S]

We need libapr-0.sl.9.sl, because this is apache lib.

> So can you make changes there too?
I am not sure how to change this one.

> Remove -lc and librwtool_v2.2 if not
> needed.

We need librwtool_v2.2 because we use tools.h.

BTW,
We use Rogue Wave library so do we need -AA or -AP?

[Dennis Handly]

>I am not sure how to change this one.

Ok.

>We need librwtool_v2.2 because we use tools.h.

What features do you use?

>We use Rogue Wave library so do we need -AA or -AP?

You need one or the other. All of the aC++ libs in the application have to agree.

[tash S]

We use we need -AA option.

>>We need librwtool_v2.2 because we use >>tools.h.
>>
> What features do you use?


We use the below.
using namespace std;
And We use the below RW objects.

#include


Untill now, I suspect that this core is just the problem of core not link issue. What do you think ?

Can you tell me 'disas' command result. What do you need the informaition in order to judge between link issue or just source issue?

[Dennis Handly]

>And we use the below RW objects.
#include

Any reason you use a RWCString vs. std::string?
Are you using any fancy member functions that aren't Standard?

>Until now, I suspect that this core is just the problem of core not link issue. What do you think?

Not sure, before we were sidetracked, I was looking to prove whether it was linkage table corruption.

>Can you tell me 'disas' command result. What do you need the information in order to judge between link issue or just source issue?

Well I need to see how it gets from _C_get_link to C_add_new_buffer but gets lost on the way.

Basically I need the disassembly so I can see that words are being used in the import stub.

> How will I fix the problem if it could be due to linkage table corruption.

Well, we set a hardware watchpoint on the location and see who modifies it. That's if you can debug apache. Or attach to it before it loads your lib.

[tash S]

Here is the dump command.

=======
(gdb) disas 0xc818fb18 0xc818fb18+4*8
Dump of assembler code from 0xc818fb18 to 0xc818fb38:
;;; File: /opt/aCC/include_std/rw/tree
;;; 405 _C_link_type _C_get_link() {
0xc818fb18 <__rw::__rb_tree,__r
w::__select1st<:pair>,RSACString *>,rw_deref_compar
e<:less>,RSACString>,std::allocator<:pair>har *>>>::_C_get_link(void)>: stw %rp,-0x14(%sp)
0xc818fb1c: ldo 0x40(%sp),%sp
0xc818fb20: stw %r19,-0x20(%sp)
0xc818fb24: nop
0xc818fb28: stw %r26,-0x64(%sp)
;;; 406 _C_link_type __tmp = _C_free_list;
0xc818fb2c: ldw -0x64(%sp),%r20
0xc818fb30: ldo 4(%r20),%r21
0xc818fb34: ldw 0(%r21),%r22
End of assembler dump.
=============

Here is apach related lib.

bash-2.04# ls -lt libapr-0.sl.9
lrwxrwxrwx 1 root sys 15 Jun 14 2006 libapr-0.sl.9@ -> liba
pr-0.sl.9.5

bash-2.04# chatr libapr-0.sl.9.5
libapr-0.sl.9.5:
shared library
shared library dynamic path search:
SHLIB_PATH disabled second
embedded path enabled first /opt/hpws/apache/lib
internal name:
libapr-0.sl.9
shared library list:
dynamic /usr/lib/librwtool_v2.2
dynamic /usr/lib/libstd_v2.2
dynamic /usr/lib/libCsup_v2.2
dynamic /usr/lib/libc.2
dynamic /usr/lib/libm.2
dynamic /usr/lib/libnsl.1
dynamic /usr/lib/libdld.2
shared vtable support disabled
static branch prediction disabled
executable from stack: D (default)
kernel assisted branch prediction enabled
lazy swap allocation disabled
text segment locking disabled
data segment locking disabled
third quadrant private data space disabled
fourth quadrant private data space disabled
third quadrant global data space disabled
data page size: D (default)
instruction page size: D (default)
nulptr references disabled
shared library private mapping disabled

============

If you know something, please let us know.

[Dennis Handly]

>Here is the dump command.
(gdb) disas 0xc818fb18 0xc818fb18+4*8

You keep giving me something that I don't want. The first time I thought I pasted it incorrectly but looking again, I had it fine.
I asked for:
(gdb) disas 0xc89cfb58 0xc89cfb58+4*8

What have you changed from the original abort? If you have changed any libs and all, you need to start all over again. We are dealing with virtual addresses and they will change if you fiddle with libs or source.

(gdb) set print max-symbolic-offset 1
(gdb) set redirect-file gdb.out
(gdb) set redirect on
(gdb) frame 1
(gdb) info reg
(gdb) disas
(gdb) set redirect off

Then attach this file so we don't bloat the thread more.

>Here is apache related lib.
# chatr libapr-0.sl.9.5
shared library list:
dynamic /usr/lib/librwtool_v2.2
dynamic /usr/lib/libstd_v2.2
dynamic /usr/lib/libCsup_v2.2
dynamic /usr/lib/libc.2
dynamic /usr/lib/libm.2
dynamic /usr/lib/libnsl.1
dynamic /usr/lib/libdld.2

Ok, this matches the rest of your ldd output.

[tash S]

Hi Dennis,

The attachement is the file you required

[Dennis Handly]

>The attachment is the file you required\

Ok, much simpler to see now. :-)

;;; 409 : (_C_next_avail == _C_last ? (_C_add_new_buffer(), _C_next_avail++)
0xc818fb8c: call 0xc818fb08 <...::_C_add_new_buffer(void)>

I need to see:
(gdb) disas 0xc818fb08 0xc818fb08+4*8

[tash S]

Hi Dennis,

Attached.

[Dennis Handly]

(gdb) disas 0xc818fb08 0xc818fb08+4*8
0xc818fb08 <::_C_add_new_buffer()>: ldw 0x1254(%r19),%r21
0xc818fb0c <+0x4>: ldw 0x1258(%r19),%r19
0xc818fb10 <+0x8>: bve (%r21)
0xc818fb14 <+0xc>: stw %rp,-0x18(%sp)

Ok, we have our PA2.0 import stub. We can check if the PLT has been corrupted.
(gdb) x /64x $r19+0x1254-8*4
(gdb) x /x $r19+0x1258

I'm assuming the last command will show that R19 is being loaded with the same value.

If not, we need to use the saved R19 value, so do this:
(gdb) x /64x *(void**)($sp-0x20)+0x1254-8*4

With those 64 words, we are looking for a pattern of lots of 0s or ascii chars, etc.

[tash S]

I attached three commands in above.
But I could not see any PLT's corrupted in the gdb result.
If we need to commands of gdb, please let me know.

[tash S]

I forgot to attach.

[tash S]

Hi Dennis,

The part of the link is following:


==========================
/opt/aCC/bin/aCC -b -o aaa.sl ...
-lapr-0 -lrwtool_v2 -lstd_v2 -lCsup_v2 -Wl,-a,archive -lwacore -lcommon -lrepository -Bimmediate -Bnonfatal
==========================

If we miss something, please let us know

Thank in advance.

[Dennis Handly]

>I attached three commands in above.
But I could not see any PLTs corrupted in the gdb result.

Right, no obvious corruption, other than PC addresses too big.

(gdb) x /64x $r19+0x1254-8*4
(gdb) x /x $r19+0x1258
>>I'm assuming the last command will show that R19 is being loaded with the same value.

R19 was changed on the call so these aren't valid.

>>we need to use the saved R19 value, so do this:
(gdb) x /64x *(void**)($sp-0x20)+0x1254-8*4

This shows that R19 was: 0x79eff728 - 0x1254 + 8*4 -> 79EFE4F4

(gdb) x /64x *(void**)($sp-0x20)+0x1254-8*4
0x79eff728: 0xc8c241c8 0x79f1cb34 0xc8c24208 0x79f1cb34
0x79eff738: 0xc8c24240 0x79f1cb34 0xc8c242f8 0x79f1cb34
0x79eff748: > 0xc8c24398 0x79f1cb34 <

The last two entries matches the current PC and R19 values. It seems we are branching from libaceauth.sl to mod_rsawa_apache.sl. Are these both your libs?
0x03ff0340 0xc76c0000 0xc778c000 0x79f1c000 0x79f2c000 0x79f1cb34

The text address range is 0xc76c0000 0xc778c000, which doesn't contain 0xc8c24398.

This is a particular kind of corruption.
1) Either there is a dld bug, what version do you have?
2) The target shlib was unloaded and another was loaded in its place. With the same data size but different code offsets.
3) Or someone incremented every other word in the PLT.

Just to make sure we match, can you do "info shared" again:
(gdb) set print max-symbolic-offset 1
(gdb) set redirect-file gdb.out
(gdb) set redirect on
(gdb) info shared
(gdb) set redirect off

Then attach this file so we don't bloat the thread more.

>The part of the link is following:
aCC -b -o aaa.sl ...
-lapr-0 -lrwtool_v2 -lstd_v2 -lCsup_v2 -Wl,-a,archive -lwacore -lcommon -lrepository
-Bimmediate -Bnonfatal

You should put the aC++ runtime libs last. Also why are you using -a,archive? And why -Bimmediate -Bnonfatal? (Did you really use -B directly or did you use -Wl,-B...?)

[tash S]

Attached "info shared" info.

[tash S]

>It seems we are branching from
>libaceauth.sl to mod_rsawa_apache.sl.
>Are these both your libs?

Yes.

>This is a particular kind of corruption.
>1) Either there is a dld bug, what version >do you have?

You mean dld?

bash-2.04# what /usr/bin/ld
/usr/bin/ld:
$Revision: 92453-07 linker linker crt0.o B.11.53 060807 $
HP aC++ B3910B A.03.73 Classic Iostream Library
HP aC++ B3910B A.03.73 Language Support Library
ld_msgs.cat: $Revision: 1.85 $
92453-07 linker command s800.sgs ld PA64 B.11.60 REL 070209


>2) The target shlib was unloaded and >another was loaded in its place. With the >same data size but different code offsets.

How to fix in this case? source code issue?


>3) Or someone incremented every other word >in the PLT.

How to fix in this case? source code issue?


> You should put the aC++ runtime libs last.

What exactly does it mean?

[Dennis Handly]

>>Are these both your libs?

>Yes.

One solution would be to combine them.
How are you telling apache that you need both libaceauth.sl and mod_rsawa_apache.sl at the same time?

>You mean dld?

Yes. B.11.60 doesn't seem that old.

>>2) The target shlib was unloaded

>How to fix in this case? source code issue?

Are you doing shl_load/shl_unload in your code?
From the values you printed:
(gdb) x /64x $r19+0x1254-8*4 mod_rsawa_apache.sl
0x79f1dd68: 0xc86620d8 0x79f1cb34

This shows the PC ranges for mod_rsawa_apache.sl are in range, 0xc86XXXXX.

But the values for libaceauth.sl seem to point to the addresses in mod_rsawa_apache.sl that are too big, 0xc8cXXXXX:
(gdb) x /64x *(void**)($sp-0x20)+0x1254-8*4 libaceauth.sl
0x79eff728: 0xc8c241c8 0x79f1cb34

So it seems at one time mod_rsawa_apache.sl was at 0xc8cXXXXX then it moved to the current value of 0xc86XXXXX. (Of course this assumes that mod_rsawa_apache.sl contains a copy of _C_add_new_buffer for that template class and not another shlib that was unloaded?)

To debug this, you would have to start apache at the beginning and use "catch load" and "catch unload" until you see what happens to mod_rsawa_apache.sl.

>>3) incremented (by a constant) every other word in the PLT.

>How to fix in this case? source code issue?

It is hard to believe anyone is doing that. Only dld knows where that data area is.

>> You should put the aC++ runtime libs last.

>What exactly does it mean?

You link line has:
-lapr-0 -lrwtool_v2 -lstd_v2 -lCsup_v2 -Wl,-a,archive -lwacore -lcommon -lrepository
-Bimmediate -Bnonfatal

It should be changed to:
-lapr-0 -lwacore -lcommon -lrepository -lrwtool_v2 -lstd_v2 -lCsup_v2
-Wl,-Bimmediate -Wl,-Bnonfatal

(In my previous message I had:
0x03ff0340 0xc76c0000 0xc778c000 0x79f1c000 0x79f2c000 0x79f1cb34
The text address range is 0xc76c0000 0xc778c000, which doesn't contain 0xc8c24398.

This was the "info shared" from the first time, your latest has:
0x03ff0340 0xc8600000 0xc86cc000 0x79f1c000 0x79f2c000 0x79f1cb34
With a text address range of 0xc8600000 0xc86cc000.)

[tash S]

>One solution would be to combine them.
>How are you telling apache that you need >both libaceauth.sl and mod_rsawa_apache.sl >at the same time?

It could be difficult to do so, because this is related to other platform compatibility.

>>You mean dld?
>Yes. B.11.60 doesn't seem that old.

Can you tell me how to download the latest dld?

>Are you doing shl_load/shl_unload in your code?
No

>To debug this, you would have to start
> apache at the beginning and use "catch
>load" and "catch unload" until you see
> what happens to mod_rsawa_apache.sl.

Is "catch load" and "catch unload" gdb command?


>It should be changed to:
> -lapr-0 -lwacore -lcommon -lrepository
> -lrwtool_v2 -lstd_v2 -lCsup_v2
> -Wl,-Bimmediate -Wl,-Bnonfatal
After the changes, I have still the same problem.

If you need "info shared", please let me know.

Thank you in advance.