1821651 Members
3244 Online
109633 Solutions
New Discussion юеВ

LD_PRELOAD and libcl.sl

 
sachin myneni
Advisor

LD_PRELOAD and libcl.sl

Hello,

I am having trouble with running my application HP-UX B.11.11.

I have an executable (a.exe) that dynamically loads a shared library (b.so). I get an undefined symbol:
/usr/lib/pa20_64/dld.sl: Unsatisfied code symbol 'FTN_DTOI' in load module '/homes/dudee/myneni/workarea/exec/b.so'

I tried setting the following environment variable at runtime:
"LD_PRELOAD=/usr/lib/pa20_64/libcl.sl"
but that doesn't help. Same message as above.

If I list "-L/usr/lib/pa20_64/ -lcl" in the link line, I get:
"Error: Cannot dlopen load module '/homes/dudee/myneni/workarea/exec//b.so' because it contains thread specific data."

At this point, I had -Wl,-aarchive, which I changed to -Wl,-ashared_archive and I get the following:
Cannot dlopen load module '/usr/lib/pa20_64/libcl.2' because it contains thread specific data.
With or without LD_PRELOAD pointing to /usr/lib/pa20_64/libcl.sl.

Can someone help me understand how to fix this issue?

Thanks in advance
-Sachin
27 REPLIES 27
Carlos Roberto Schimidt
Regular Advisor

Re: LD_PRELOAD and libcl.sl

Hi,

Try put path from your shared library in SHLIB_PATH enviroment variable.

Schimidt
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Hi Sachin,

Could you provide more information about your server's architecture etc...
# uname -a
# file a.exe

thx
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Here is the runtime environment and the output of file on both the executable and the shared object:

myneni@hp76 BOM_opti/os-mbd-fem > uname -a
HP-UX hp76 B.11.11 U 9000/785 3968680693 unlimited-user license
myneni@hp76 BOM_opti/os-mbd-fem > file ~/os_binaries/os80d_HP-UX_42_64mp
/homes/dudee/t/tad/os_binaries/os80d_HP-UX_42_64mp: ELF-64 executable object file - PA-RISC 2.0 (LP64)
myneni@hp76 BOM_opti/os-mbd-fem > file ~/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so
/homes/dudee/myneni/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so: ELF-64 shared object file - PA-RISC 2.0 (LP64)

myneni@hp76 BOM_opti/os-mbd-fem > chatr ~/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so
/homes/dudee/myneni/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so:
64-bit ELF shared library
shared library dynamic path search:
LD_LIBRARY_PATH enabled first
SHLIB_PATH enabled second
embedded path enabled third /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs:/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX:/opt/fortran90/lib/pa20_64:/usr/lib/pa20_64:/opt/aCC/lib/pa20_64:/usr/lib/pa20_64:/usr/lib/pa20_64:/opt/langtools/lib/pa20_64
shared vtable support disabled
segments:
index type address flags size
5 text 4000000000000000 z---c D (default)
6 data 8000000100000000 ---m- D (default)
kernel assisted branch prediction enabled
lazy swap allocation for dynamic segments disabled
myneni@hp76 BOM_opti/os-mbd-fem >

Here is the information on the build machine:

myneni@hp77 p4_mbd/mbd > uname -a
HP-UX hp77 B.11.11 U 9000/785 2005242756 unlimited-user license
myneni@hp77 p4_mbd/mbd >


Thanks
-Sachin


Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Hi Sachin,

How about the output of the following commands...

# file a.exe
# ldd -v a.exe
# file /homes/dudee/myneni/workarea/exec/b.so
# ldd -v /homes/dudee/myneni/workarea/exec/b.so
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Hi Sachin,

How about the output of the following commands...

# file a.exe
# ldd -v a.exe
# file /homes/dudee/myneni/workarea/exec/b.so
# ldd -v /homes/dudee/myneni/workarea/exec/b.so

...also post the command used for compilation.
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Hi Sandman,

I don't have ldd on any of my HP-UX systems, so I cannot give the output of 2 of the commands. But here is the output of 'file' from both the executable and shared object:
Executable:
myneni@hp76 BOM_opti/os-mbd-fem > file ~tad/os_binaries/os80d_HP-UX_42_64mp
/homes/dudee/t/tad/os_binaries/os80d_HP-UX_42_64mp: ELF-64 executable object file - PA-RISC 2.0 (LP64)
Shared Object:
myneni@hp76 BOM_opti/os-mbd-fem > file /homes/dudee/myneni/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so
/homes/dudee/myneni/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so: ELF-64 shared object file - PA-RISC 2.0 (LP64)

COMPILATION FLAGS:
FORTRAN compilation flags: /opt/fortran90/bin/f90 +U77 +DA2.0W +DS2.0 -Df90 +O2 +Z +Onoopenmp +source=default +noextend_source
C++ Compilation flags: /opt/aCC/bin/aCC +DA2.0W +DS2.0 -Dhpf90 -Dhpux +O2 +Z
Verbose Link output:

/opt/aCC/bin/aCC +DA2.0W +DS2.0 -Wl,-aarchive,+vnocompatwarnings -v //OBJ_LIST// +O2 -b -L/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs -L/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX -lnumerflib_HP-UX_64mp -lnumerclib_HP-UX_64mp -lblas_HP-UX_64mp -lblas_HP-UX_64mp -L/opt/fortran90/lib/pa20_64 -lU77 -lF90_parallel -L/usr/lib/pa20_64 -lstd -lstream -lCsup -lm -L/opt/aCC/lib/pa20_64 -ldemangle -lcl /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/cflex_64.o /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/lm_new_cflex_64.o -llmgr_cflex_64 -lcrvs_cflex_64 -lsb_cflex_64 -L/usr/lib/pa20_64 -o nusolver.so
LPATH=/usr/lib/pa20_64:/opt/langtools/lib/pa20_64
/usr/ccs/bin/ld -b -o /tmp/nusolver.so_myneni +I__shlTerm -aarchive +vnocompatwarnings acceleration.o ...... zipother.o -L /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs -L /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX -lnumerflib_HP-UX_64mp -lnumerclib_HP-UX_64mp -lblas_HP-UX_64mp -lblas_HP-UX_64mp -L /opt/fortran90/lib/pa20_64 -lU77 -lF90_parallel -L /usr/lib/pa20_64 -lstd -lstream -lCsup -lm -L /opt/aCC/lib/pa20_64 -ldemangle -lcl /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/cflex_64.o /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/lm_new_cflex_64.o -llmgr_cflex_64 -lcrvs_cflex_64 -lsb_cflex_64 -L /usr/lib/pa20_64 >/var/tmp/AAAa17623 2>&1 removing /var/tmp/AAAa17623 nusolver.so done




The following are the compilation flags for the executable (optistruct.exe) that loads the shared object built above:
FORTRAN: /opt/fortran90/bin/f90 +U77 +DA2.0W +DS2.0 -Df90 +O3 +Onoopenmp +source=default +noextend_source
C : cc +DA2.0W +DS2.0 -Dhpf90 -Dhpux +O3
C++ : /opt/aCC/bin/aCC +DA2.0W +DS2.0 -Dhpf90 -Dhpux +O3
LD: : /opt/aCC/bin/aCC +DA2.0W +DS2.0 -Wl,-aarchive,+vnocompatwarnings +O3 -s +Oparallel


Thank you
-Sachin


sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

That was quite unreadable. Sorry. Here goes again:

Hi Sandman,

I don't have ldd on any of my HP-UX systems, so I cannot give the output of 2 of the commands. But here is the output of 'file' from both the executable and shared object:

Executable:
-----------
myneni@hp76 BOM_opti/os-mbd-fem > file ~tad/os_binaries/os80d_HP-UX_42_64mp
/homes/dudee/t/tad/os_binaries/os80d_HP-UX_42_64mp: ELF-64 executable object file - PA-RISC 2.0 (LP64)

Shared Object:
--------------
myneni@hp76 BOM_opti/os-mbd-fem > file /homes/dudee/myneni/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so
/homes/dudee/myneni/workarea/p4_mbd/mbd/exec/HP-UX_64mp/nusolver.so: ELF-64 shared object file - PA-RISC 2.0 (LP64)


COMPILATION FLAGS:
==================
FORTRAN compilation flags:
--------------------------
/opt/fortran90/bin/f90 +U77 +DA2.0W +DS2.0 -Df90 +O2 +Z +Onoopenmp +source=default +noextend_source

C++ Compilation flags:
----------------------
/opt/aCC/bin/aCC +DA2.0W +DS2.0 -Dhpf90 -Dhpux +O2 +Z


Verbose Link output:
--------------------
/opt/aCC/bin/aCC +DA2.0W +DS2.0 -Wl,-aarchive,+vnocompatwarnings -v //OBJ_LIST// +O2 -b -L/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs -L/a/homes/dudee/m
/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX -lnumerflib_HP-UX_64mp -lnumerclib_HP-UX_64mp -lblas_HP-UX_64mp -lblas_HP-UX_64mp -L/opt/fortran90/lib/pa20_64 -lU77 -lF90_parallel -L/usr/lib/pa20_64 -lstd -lstream -lCsup -lm -L/opt/aCC/lib/pa20_64 -ldemangle -lcl /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/cflex_64.o
/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/lm_new_cflex
_64.o -llmgr_cflex_64 -lcrvs_cflex_64 -lsb_cflex_64 -L/usr/lib/pa20_64 -o nusolver.so
LPATH=/usr/lib/pa20_64:/opt/langtools/lib/pa20_64
/usr/ccs/bin/ld -b -o /tmp/nusolver.so_myneni +I__shlTerm -aarchive +vnocompatwarnings acceleration.o ...... zipother.o -L /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs -L /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX -lnumerflib_
HP-UX_64mp -lnumerclib_HP-UX_64mp -lblas_HP-UX_64mp -lblas_HP-UX_64mp -L /opt/fortran90/lib/pa20_64 -lU77 -lF90_parallel -L /usr/lib/pa20_64 -lstd -lstream -lCsup -lm -L /opt/aCC/lib/pa20_64 -ldemangle -lcl /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/cflex_64.o
/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/lm_new_cflex
_64.o -llmgr_cflex_64 -lcrvs_cflex_64 -lsb_cflex_64 -L /usr/lib/pa20_64 >/var/tmp/AAAa17623 2>&1 removing /var/tmp/AAAa17623 nusolver.so done





The following are the compilation flags for the executable (optistruct.exe) that loads the shared object built above:

FORTRAN: /opt/fortran90/bin/f90 +U77 +DA2.0W +DS2.0 -Df90 +O3 +Onoopenmp +source=default +noextend_source

C : cc +DA2.0W +DS2.0 -Dhpf90 -Dhpux +O3

C++ : /opt/aCC/bin/aCC +DA2.0W +DS2.0 -Dhpf90 -Dhpux +O3

LD: : /opt/aCC/bin/aCC +DA2.0W +DS2.0 -Wl,-aarchive,+vnocompatwarnings +O3 -s +Oparallel


Thank you
-Sachin
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Hi Sachin,

Thanks for the legible output. From your last post it looks like you have the wrong library path for linking the "libcl.2" library (that's if I'm reading the aCC compilation correctly).

Under your "Verbose Link output" section the lib path is specified...

"-L/opt/aCC/lib/pa20_64 -ldemangle -lcl"

...which is incorrect since the libcl.sl is found under "/usr/lib/pa20_64". In fact you can search for it as:

# find / -name "libcl.sl" -type f
<...should show up only in /usr/lib/pa20_64>

Another suggestion would be not to use "-Wl,-aarchive" and let the defaults apply. The default action is to link the shared lib (and if it's not found bind the executable with the archive version). See ld(1) for details.

cheers!
Dennis Handly
Acclaimed Contributor

Re: LD_PRELOAD and libcl.sl

You first need to explain what language(s) are in a.exe and what language(s) are in b.so.

It seems that a.exe is C++ and b.so is Fortran?
If you are doing that, you should link a.exe with libcl.sl.

A similar issue occurs with C and aC++ and we have a nice web explaining what to do:
http://docs.hp.com/en/7133/distributing.htm

I would have expected your first LD_PRELOAD to work, though I would have used .../libcl.2.

Your second attempt, -lcl will not work unless you have the LD_PRELOAD too.

You said you had -Wl,-aarchive, where?
As Sandman suggests, you should using the default, especially when creating shlibs.
From your link line of a.exe, that's your problem! If you insist upon using archives, you must have the system libs as shared.
You can use -l:libm.a for performance.

Note you seem to be including all of the aC++ libs? Why don't they appear twice in the aCC -v output? (Ah, aCC -b.)
In particular, you should have -ldemangle
before the default libs. And there should be no user libs/objects after them.
cflex_64.o...-lsb_cflex_64

>I don't have ldd on any of my HP-UX systems

You should have /usr/ccs/bin/ldd on your system. If not, you need a newer linker patch. (And we really need that ldd or chatr output.)

Also for aCC, you should change +DA2.0W to +DD64.

A comment about Sandman's comment:
-L/opt/aCC/lib/pa20_64 -ldemangle -lcl"
...which is incorrect

This isn't incorrect since -L paths are additive and -lcl will be found under the default or previous -L paths.
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Dennis,

Here are the respnses to your various questions/suggestions:

Language:
=========
a.exe: Mixed FORTRAN, C and C++. Linked using aCC.
b.so: Mixed FORTRAN, C and C++. Linked using aCC.


myneni@hp77 p4_mdl/os > aCC -Version
aCC: HP ANSI C++ B3910B A.03.55
myneni@hp77 p4_mdl/os > f90 +version
HP F90 v2.5.1


LD_PRELOAD:
==========
I tried linking a.exe with -Wl,-ashared,-lcl
Also tried LD_PRELOAD=libcl.2 and LD_PRELOAD=/usr/lib/pa20_64/libcl.sl and LD_PRELOAD=/usr/lib/pa20_64/libcl.2

LDD:
====
Thanks for pointing out the location of LDD. Here is the output.

myneni@hp77 p4_mdl/os > /usr/ccs/bin/ldd exec/os80d_HP-UX_42_64mp
libm.2 => /usr/lib/pa20_64/libm.2
libpthread.1 => /usr/lib/pa20_64/libpthread.1
libcl.2 => /usr/lib/pa20_64/libcl.2
libdl.1 => /usr/lib/pa20_64/libdl.1
libdl.1 => /usr/lib/pa20_64/libdl.1
/opt/graphics/OpenGL/lib/pa20_64/libogltls.sl => /opt/graphics/OpenGL/lib/pa20_64/libogltls.sl
libc.2 => /usr/lib/pa20_64/libc.2
libdl.1 => /usr/lib/pa20_64/libdl.1

myneni@hp77 p4_mdl/os > /usr/ccs/bin/ldd ~/nusolver/80/HPUX_64/nusolver.so
libstd.2 => /usr/lib/pa20_64/libstd.2
libstream.2 => /usr/lib/pa20_64/libstream.2
libCsup.2 => /usr/lib/pa20_64/libCsup.2
libm.2 => /usr/lib/pa20_64/libm.2
myneni@hp77 p4_mdl/os >

I am not sure what we need the OpenGL library for since the application is a command line app. I will need to look into that.


Here's the CHATR output:
========================
myneni@hp77 p4_mdl/os > chatr exec/os80d_HP-UX_42_64mp
exec/os80d_HP-UX_42_64mp:
64-bit ELF executable
shared library dynamic path search:
LD_LIBRARY_PATH enabled first
SHLIB_PATH enabled second
embedded path enabled third /opt/aCC/lib/pa20_64:/a/homes/dudee/m/myneni/workarea/p4_mdl/os/os_libs:/a/homes/dudee/m/myneni/workarea/p4_mdl/os/OSLIBS/LIB-HP-UX:/opt/mlib/lib/pa20_64/:/opt/fortran90/lib/pa20_64:/usr/lib/pa20_64:/usr/lib/pa20_64:/opt/langtools/lib/pa20_64
shared library list:
libm.2
libpthread.1
libcl.2
libdl.1
shared library binding:
deferred
global hash table disabled
global hash table size 1103
shared library mapped private disabled
shared library segment merging disabled
shared vtable support disabled
explicit unloading disabled
segments:
index type address flags size
6 text 4000000000000000 z---c- D (default)
7 data 8000000100000000 ---m-- D (default)
executable from stack: D (default)
static branch prediction disabled
kernel assisted branch prediction enabled
lazy swap allocation for dynamic segments disabled
nulptr references disabled
myneni@hp77 p4_mdl/os >

myneni@hp77 p4_mdl/os > chatr ~/nusolver/80/HPUX_64/nusolver.so
/homes/dudee/myneni/nusolver/80/HPUX_64/nusolver.so:
64-bit ELF shared library
shared library dynamic path search:
LD_LIBRARY_PATH enabled first
SHLIB_PATH enabled second
embedded path enabled third /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs:/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX:/opt/fortran90/lib/pa20_64:/usr/lib/pa20_64:/opt/aCC/lib/pa20_64:/usr/lib/pa20_64:/opt/langtools/lib/pa20_64
shared library list:
libstd.2
libstream.2
libCsup.2
libm.2
shared vtable support disabled
explicit unloading enabled
segments:
index type address flags size
5 text 4000000000000000 z---c- D (default)
6 data 8000000100000000 ---m-- D (default)
static branch prediction disabled
kernel assisted branch prediction enabled
lazy swap allocation for dynamic segments disabled
nulptr references disabled
myneni@hp77 p4_mdl/os >



Archive versus Shared:
======================
Understood. I tried:
-Wl,-a,archive,-L/opt/fortran90/lib/pa20_64 -lU77 -lF90_parallel -Wl,-a,shared,-L/usr/lib/pa20_64 -lcl -lpthread -Wl,-a,archive

Thank you.
-Sachin
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Sachin,

Reading your posts again, I'm wondering why you are not linking your Fortran executable with the shared library routines provided under "/opt/fortran90/lib/pa20_64". Since your error is coming from the compiled Fortran code and not the C/C++ executable. Suggestion is...

/opt/fortran90/bin/f90 +U77 +DA2.0W +DS2.0 -Df90 +O2 +Z +Onoopenmp +source=default +noextend_source -L/opt/fortran90/lib/pa20_64

hope it helps!
Dennis Handly
Acclaimed Contributor

Re: LD_PRELOAD and libcl.sl

If you have aC++ code, you should really be compiling and linking with -mt, since you use libpthread. (Unless you don't use iostreams nor strings.)

You neglected to mention that it now works?
Or what the new error is.

With libcl.2 there, you shouldn't need LD_PRELOAD.

The OpenGL lib is added by default by dld.

>Understood. I tried:
-Wl,-a,archive,-L/opt/fortran90/lib/pa20_64 -lU77 -lF90_parallel -Wl,-a,shared,-L/usr/lib/pa20_64 -lcl -lpthread -Wl,-a,archive

You can not do this. You have both shared and archive versions of libs. Remove the -Wl,-a,archive at the end. You want system libs as shared. You can also remove "-L/usr/lib/pa20_64 -lcl"

In fact you should remove all -Wl,-a and use -l:libU77.a -l:libF90_parallel.a. And possibly add -l:libm.a.
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Sandman,

I do use "/opt/fortran90/lib/pa20_64"

-Sachin
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Hi Sandman,

I wanted to add to my previous post:

We have been unable to link with f90 because of C and C++ in our source. Since C++ was introduced into our mostly FORTRAN code, we have been linking with aCC.

-Sachin
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Hello Dennis,

My problem isn't solved yet:( I tried everything suggested (and combinations of everything suggested)

We use +Oparallel with our linker. We found that some parts of the code, when compiled with +Oparallel, were very ineffecient. And since most of the time is spent in a different, solver module, we no longer compile with +Oparallel, but we do use this directive in linking. I do believe we need the +Oparallel to allow multithreading in veclib.

I tried -l:libU77.a format:

/opt/aCC/bin/aCC +DA2.0W +DS2.0 -mt -Wl,-B,immediate,+vnocompatwarnings,+s,+fb //OBJ_LIST// +O3 -s +Oparallel -L/a/homes/dudee/m/myneni/workarea/p4_mdl/os/os_libs -L/a/homes/dudee/m/myneni/workarea/p4_mdl/os/OSLIBS/LIB-HP-UX -losbase_HP-UX_64mp -losinput_HP-UX_64mp -losfea_HP-UX_64mp -lossolver_HP-UX_64mp -losmbd_HP-UX_64mp -loshfs_HP-UX_64mp -losopti_HP-UX_64mp -losresults_HP-UX_64mp -losmbd_HP-UX_64mp -lossensi_HP-UX_64mp -los3rdparty_HP-UX_64mp -losfea_HP-UX_64mp -lossolver_HP-UX_64mp -loshfs_HP-UX_64mp -losopti_HP-UX_64mp -lossensi_HP-UX_64mp -losinput_HP-UX_64mp -lbcslib42_HP-UX_64mp -lmetis400_11 -L/opt/mlib/lib/pa20_64/ -l:libveclib.a -lh3dwriter8_HP-UX_64 -ljpeg_11 -lzlib_11 -lblas_HP-UX_64mp -L/usr/lib/pa20_64 -L/opt/fortran90/lib/pa20_64 /usr/lib/pa20_64/libcps.a -l:libm.a -l:libU77.a -l:libF90_parallel.a -L/usr/lib/pa20_64 -lpthread -o os80d_HP-UX_42_64mp
os80d_HP-UX_42_64mp done

Here's the LDD output:
=======================
myneni@hp77 p4_mdl/os > ldd exec/os80d_HP-UX_42_64mp
libpthread.1 => /usr/lib/pa20_64/libpthread.1
libstd.2 => /usr/lib/pa20_64/libstd.2
libstream.2 => /usr/lib/pa20_64/libstream.2
libCsup.2 => /usr/lib/pa20_64/libCsup.2
libcps.2 => /usr/lib/pa20_64/libcps.2
libm.2 => /usr/lib/pa20_64/libm.2
libcl.2 => /usr/lib/pa20_64/libcl.2
libc.2 => /usr/lib/pa20_64/libc.2
libdl.1 => /usr/lib/pa20_64/libdl.1
libdl.1 => /usr/lib/pa20_64/libdl.1
libdl.1 => /usr/lib/pa20_64/libdl.1
/opt/graphics/OpenGL/lib/pa20_64/libogltls.sl => /opt/graphics/OpenGL/lib/pa20_64/libogltls.sl
myneni@hp77 p4_mdl/os >

Strangely libm.a is not statically bound.
Also note that I have also tried +fb.

Here is the output from my run (same with or without LD_PRELOAD=libcl.sl or any variant thereof):

.....
Number of dependent coordinates : 6
Number of independent coordinates: 6

Starting dynamic analysis ...

Time=1.000E-06; H=1.000E-06 [Max Phi=0.000E+00]/usr/lib/pa20_64/dld.sl: Unsatisfied code symbol 'FTN_DTOI' in load module '/homes/dudee/myneni/nusolver/80/HPUX_64/nusolver.so'.
sh: 12552 Abort

....


It might seem obvious that I gravitate towards this possibility now;) but is it possible that either dld.sl or ld or both could be the culprits?

What should I look for?

Thank you for your suggestions so far. I still hope to get this solved. I need to :0

-Sachin
Dennis Handly
Acclaimed Contributor

Re: LD_PRELOAD and libcl.sl

>I do believe we need the +Oparallel to allow multithreading in veclib.

You could link with -lpthread -lcps.

> I tried -l:libU77.a format:
> -l:libm.a -l:libU77.a -l:libF90_parallel.a

You should place libm last in this set.

> Here's the LDD output:
> Strangely libm.a is not statically bound.

There are really two libms. One for -l:libm.a and one -lm tacked on by aCC. This probably not a problem.

> Also note that I have also tried +fb.

Does it work without it?

>Unsatisfied code symbol 'FTN_DTOI' in load module '/homes/dudee/myneni/nusolver/80/HPUX_64/nusolver.so'.
> but is it possible that either dld.sl or ld or both could be the culprits?

Perhaps. This seems strange. You have libcl in your ldd output. (At least this explains why LD_PRELOAD doesn't work. ;-)
What version of dld.sl and libcl.2 do you have?

On my system I have FTN_DTOI there. Can you do:
$ elfdump -s -n .dynsym /usr/lib/pa20_64/libcl.sl | fgrep FTN_DTOI

If you see a result, then libcl.2 is fine and the problem might be in dld.sl.

You might also try chatr -Bimmediate on your executable and then rerun. Perhaps dld isn't printing all of the unsats and we are only looking at the first???

Of course you can just give up and extract FTN_DTOI (fd_dtoi.o) from libcl.a and link nusolver.so with it. ;-)
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Hi Sachin,

Let's try to simplify things a bit since there is a lot of information and a step-by-step analysis would make things clearer (I hope).

It would help if you supplied the "+vshlibunsats" switch to the aCC compiler in order to figure out unresolved symbols in the libraries themselves. Also make sure that libcl.sl is the last one specified in order to get the link order correct. Currently I'm not seeing that in your compilation command (unless I am not reading the posted output correctly).

Please post the output of the compilation with the above modifications and report errors if any.

thanks!
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Hi Sandman,

My aCC does not seem to recognize this option (+vshlibunsats)

myneni@hp77 p4_mdl/os > aCC -V
aCC: HP ANSI C++ B3910B A.03.55

I agree that we should take thi step-by-step. I would like to follow one path (your's, Dennis's ...doesn't matter) till I reach a conclusion. And if that doesn't work start over again on another path.

There are some constraints though. Example: I cannot easily upgrade the compilers on my build system. That's out of my control. I could try getting our systems guys to upgrade other run-time libraries on the target machines though. But that itself might take some coaxing:(

So where do you suggest we begin?

Thank you
-Sachin

sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

>> I tried -l:libU77.a format:
>> -l:libm.a -l:libU77.a -l:libF90_parallel.a

>You should place libm last in this set.

Done.

>> Also note that I have also tried +fb.

>Does it work without it?

No.

>>Unsatisfied code symbol 'FTN_DTOI' in load module '/homes/dudee/myneni/nusolver/80/HPUX_64/nusolver.so'.
>> but is it possible that either dld.sl or ld or both could be the culprits?

>Perhaps. This seems strange. You have libcl in your ldd output. (At least this explains why LD_PRELOAD doesn't work. ;-)
>What version of dld.sl and libcl.2 do you have?

How do I find that?


>On my system I have FTN_DTOI there. Can you do:
>$ elfdump -s -n .dynsym /usr/lib/pa20_64/libcl.sl | fgrep FTN_DTOI

>If you see a result, then libcl.2 is fine and the problem might be in dld.sl.

On the target machine:

myneni@hp76 BOM_opti/os-mbd-fem > elfdump -s -n .dynsym /usr/lib/pa20_64/libcl.sl | fgrep FTN_DTOI
170 FUNC GLOB 0 0xf 0x800000000003ea90 0 FTN_DTOI

On the build machine:
myneni@hp77 dudee/myneni > elfdump -s -n .dynsym /usr/lib/pa20_64/libcl.sl | fgrep FTN_DTOI
170 FUNC GLOB 0 0xf 0x800000000003ea90 0 FTN_DTOI

So libcl.sl seems to be fine. How do I check dld.sl?

>You might also try chatr -Bimmediate on your executable and then rerun. Perhaps dld isn't printing all of the unsats and we are only looking at the first???

Doen't work.


>Of course you can just give up and extract FTN_DTOI (fd_dtoi.o) from libcl.a and link nusolver.so with it. ;-)

:) I really hope not to have to do that

Thanks
-Sachin
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Is my dld.sl too old? I am thinking it is.

myneni@hp76 dudee/myneni > what /usr/lib/dld.sl
/usr/lib/dld.sl:
SMART_BIND
92453-07 dld dld dld.sl B.11.18 000922


Can anyone point me to the patches that I need to install to updated dld.sl?

Thank you
-Sachin
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Hi Sachin,

I forgot to mention that the "+vshlibunsat" is an option for the linker not the compiler. Do as follows:

# aCC -Wl,+vshlibunsat -lcl

I mean to say add "+vshlibunsat" switch alongwith other switches you are giving on the aCC command line. Also, include libcl as the last switch to your aCC command line (as shown above) so that shlibs dependent on the FTN_DTOI function can resolve it externally. Post the output you get.

cheers!
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

I tried it but looks like my linked doesn't recognize that option:

ld: Unrecognized argument: +vshlibunsat
Fatal error.
make: *** [link_HP-UX_11_blas] Error 1
myneni@hp77 p4_mdl/os > ld -V
92453-07 linker command s800.sgs ld PA64 B.11.38 REL 031217
ld: 92453-07 linker linker ld B.11.38 031218
ld: Usage: ld [options] [flags] files
myneni@hp77 p4_mdl/os >


-Sachin
Sandman!
Honored Contributor

Re: LD_PRELOAD and libcl.sl

Well in that case link it with "-lcl" as the last switch on your command line for resolving external symbols and by the way what is this file (some details would be helpful):

>>/homes/dudee/myneni/nusolver/80/HPUX_64/nusolver.so
sachin myneni
Advisor

Re: LD_PRELOAD and libcl.sl

Hello Sandman,

I have 1 executable which runs another exexcutable through system() call and that executable in turn loads a shared object.

Here is the link-line of the executable that I modified to contain -lcl at the very end. Also note that I no longer have -Wl,-a,archive:

Linking executable os80d_HP-UX_42_64mp 0309388_3806000_H11_blasBW42M
/opt/aCC/bin/aCC +DA2.0W +DS2.0 -Wl,+vnocompatwarnings //OBJ_LIST// +O3 -s +Oparallel -L/a/homes/dudee/m/myneni/workarea/p4_mdl/os/os_libs -L/a/homes/dudee/m/myneni/workarea/p4_mdl/os/OSLIBS/LIB-HP-UX -losbase_HP-UX_64mp -losinput_HP-UX_64mp -losfea_HP-UX_64mp -lossolver_HP-UX_64mp -losmbd_HP-UX_64mp -loshfs_HP-UX_64mp -losopti_HP-UX_64mp -losresults_HP-UX_64mp -losmbd_HP-UX_64mp -lossensi_HP-UX_64mp -los3rdparty_HP-UX_64mp -losfea_HP-UX_64mp -lossolver_HP-UX_64mp -loshfs_HP-UX_64mp -losopti_HP-UX_64mp -lossensi_HP-UX_64mp -losinput_HP-UX_64mp -lbcslib42_HP-UX_64mp -lmetis400_11 -L/opt/mlib/lib/pa20_64/ -l:libveclib.a -lh3dwriter8_HP-UX_64 -ljpeg_11 -lzlib_11 -lblas_HP-UX_64mp -L/opt/fortran90/lib/pa20_64 -l:libU77.a -l:libF90_parallel.a -l:libm.a -L/usr/lib/pa20_64 -lpthread -lcl -o os80d_HP-UX_42_64mp
os80d_HP-UX_42_64mp done


nusolver.so is a shared object that is called indirectly by my executable:
The executable [os80d_HP-UX_42_64mp] makes a system call [system(osm_solve.exe)] and this osm_solve.exeuses dlopen() to load the shared object [nusolver.so]

Here is the link-line of nusolver.so. Once again, no -Wl,-a,archive and -lcl at the end:

Linking executable nusolver.so 0309388_3806000_H11_blasWM
/opt/aCC/bin/aCC +DA2.0W +DS2.0 -Wl,+vnocompatwarnings //OBJ_LIST// +O2 -b -L/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/nu_libs -L/a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX -lnumerflib_HP-UX_64mp -lnumerclib_HP-UX_64mp /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/cflex_64.o /a/homes/dudee/m/myneni/workarea/p4_mbd/mbd/OSLIBS/LIB-HP-UX/lm_new_cflex_64.o -llmgr_cflex_64 -lcrvs_cflex_64 -lsb_cflex_64 -lblas_HP-UX_64mp -lblas_HP-UX_64mp -L/opt/fortran90/lib/pa20_64 -l:libU77.a -l:libF90_parallel.a -L/usr/lib/pa20_64 -lstd -lstream -lCsup -l:libm.a -L/opt/aCC/lib/pa20_64 -ldemangle -lcl -o nusolver.so
nusolver.so done


With this setup, I get the following (expected?) error on the target machine:
Error: Cannot dlopen load module '/usr/lib/pa20_64/libcl.2' because it contains thread specific data.

Now I should set LD_PRELOAD=libcl.sl or LD_PRELOAD=libcl.2 or LD_PRELOAD=/usr/lib/pa20_64/libcl.sl and it is supposed to work. But I keep getting the same error:(
libcl.s->libcl.2 exist and it contains the symbold FTN_DTOI because of which I need the shared library libcl.sl .

Thanks
-Sachin