topic Shared Library Order in Linux? Executable misbehaving due to wrong Library.... in Operating System - Linux

Shared Library Order in Linux? Executable misbehaving due to wrong Library....

Alzhy — Wed, 18 Nov 2009 14:40:07 GMT

I'ev a tool (an HBA utility) that no longer works after a suite of monitoring tools were installed. LDD analysis follows below. I reckon I've now to explicitly set LD_LIBRARY_PATH? Or is there a different config file to spec the hierarchy of the libraries?

BEFORE:
# ldd scli
linux-gate.so.1 => (0xffffe000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x00b13000)
libm.so.6 => /lib/libm.so.6 (0xf7f09000)
libqlsdm.so => /usr/lib/libqlsdm.so (0xf7e8a000)
libHBAAPI.so => /usr/lib/libHBAAPI.so (0xf7e7d000)
libqsw.so => /usr/lib/libqsw.so (0xf7deb000)
libqswstubssl.so => /usr/lib/libqswstubssl.so (0xf7de8000)
libqswopenssl.so => /usr/lib/libqswopenssl.so (0xf7d24000)
libpthread.so.0 => /lib/libpthread.so.0 (0xf7d0d000)
libc.so.6 => /lib/libc.so.6 (0x009cb000)
/lib/ld-linux.so.2 (0x009ad000)
libdl.so.2 => /lib/libdl.so.2 (0xf7d09000)

AFTER(tool no longer works):
# ldd scli
linux-gate.so.1 => (0xffffe000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x00b13000)
libm.so.6 => /lib/libm.so.6 (0xf7ec3000)
libqlsdm.so => /opt/ibm/icc/cimom/lib/libqlsdm.so (0xf7e8b000)
libHBAAPI.so => /opt/ibm/icc/cimom/lib/libHBAAPI.so (0xf7e7e000)
libqsw.so => /opt/ibm/icc/cimom/lib/libqsw.so (0xf7e0f000)
libqswstubssl.so => /opt/ibm/icc/cimom/lib/libqswstubssl.so (0xf7e0c000)
libqswopenssl.so => /opt/ibm/icc/cimom/lib/libqswopenssl.so (0xf7d48000)
libpthread.so.0 => /lib/libpthread.so.0 (0xf7d31000)
libc.so.6 => /lib/libc.so.6 (0x009cb000)
/lib/ld-linux.so.2 (0x009ad000)
libdl.so.2 => /lib/libdl.so.2 (0xf7d2d000)

# # scli
Scanning QLogic FC HBA(s) and device(s), please wait...
scli: symbol lookup error: scli: undefined symbol: qla_get_os_device_name_by_handle

TIA!

Re: Shared Library Order in Linux? Executable misbehaving due to wrong Library....

Steven E. Protter — Wed, 18 Nov 2009 16:02:33 GMT

Shalom again,

Coming up with a search path that works for everything would lead me to pull my hair out.

Far easier, and more interesting things to do. Set the LD_LIBRARY_PATH in a shell script that invokes the utility.

SEP

Re: Shared Library Order in Linux? Executable misbehaving due to wrong Library....

Matti_Kurkela — Wed, 18 Nov 2009 16:16:26 GMT

The configuration file is:

/etc/ld.so.conf

It is basically a list of directories, one per line. In newer Linux distributions it may contain an "include" directive, e.g.:

include /etc/ld.so.conf.d/*.conf

In that case, you'll also have the option of putting your customizations in a configuration file you can name as you wish, e.g. /etc/ld.so.conf.d/custom.conf.
You should check your Linux distribution's configuration file naming policy, to avoid filename conflicts when you install new software or upgrade.

After making any changes to /etc/ld.so.conf or /etc/ld.so.conf.d/, you should run "ldconfig" to re-create /etc/ld.so.cache.

Use the commands "man ldconfig" and "man ld.so" for more detailed information.

MK

Re: Shared Library Order in Linux? Executable misbehaving due to wrong Library....

Alzhy — Wed, 18 Nov 2009 16:42:40 GMT

Matti,

Yeppers indeed it is /etc/ld.so.conf and it now has some includes. Man on ldd actually gave me those 2 tools ldconfig and ld.so but I am still confused whether to have the order and hierarchy of the libs globally modded or follow the standard tweak (as I've used in other UNIX dialects) and that is to have script wrappers with the correct LD_LIBRARY_PATH spec.

# more /etc/ld.so.conf
include ld.so.conf.d/*.conf

and indeed under /etc/ld.so.conf.d/

ibmld.conf qt-x86_64.conf

So are these LIBs placed ahead of /lib64, /lib and /usr/lib? There is no LD_LIBRARY_PATH set after the install that broke my HBA tool btw. How is the order set my ldconfig sir? (I've yet to read ldconfig)

Re: Shared Library Order in Linux? Executable misbehaving due to wrong Library....

Donny Jekels — Wed, 18 Nov 2009 20:05:10 GMT

ldconfig -p

Re: Shared Library Order in Linux? Executable misbehaving due to wrong Library....

Matti_Kurkela — Wed, 18 Nov 2009 22:37:53 GMT

> include ld.so.conf.d/*.conf

This include directive reads the files in standard alpha-numeric sorting order.

Your IBM monitoring software that broke your HBA tool probably installed the ibmld.conf file into /etc/ld.so.conf.d/.

The directories /usr/lib and /lib are called "trusted" library directories. When a program is loaded, LD_LIBRARY_PATH takes the highest priority, if it's set. After that, /etc/ld.so.cache is consulted. As the last step, if the library is still not found, or if the ld.so.cache file is not available, /usr/lib and /lib are checked.

"ldconfig" is the command that builds /etc/ld.so.cache. It examines the directories listed in /etc/ld.so.conf file (and the files listed in include directives within it), then /usr/lib and /lib.

My system has a bit different filename, but I'd guess that the /lib64 directory is specified in your /etc/ld.so.conf.d/qt-x86_64.conf file.

The end result is that /usr/lib and /lib are always the _last_ places to look for libraries, unless they have been explicitly listed earlier in the configuration.

So, your current library search path would probably be:

1.) $LD_LIBRARY_PATH (which is usually empty)
2.) the directories listed in ibmld.conf (=probably at least /opt/ibm/icc/cimom/lib)
3.) /lib64 (from qt-x86_64.conf, I guess)
4.) /usr/lib (built-in default)
5.) /lib (built-in default)

If my guess about the ibmld.conf file is correct, I think it's quite pretentious for IBM to assume that the special libraries of their very special product are good for everyone else too.

I would consider removing ibmld.conf from the system-wide library search path configuration, and adding an "export LD_LIBRARY_PATH=..." setting that has the same effect as the original ibmld.conf to the startup scripts of the monitoring tool suite. That way the changed library path is effective for the IBM monitoring tool suite and its child processes only, and the rest of the system will not be affected.

Making a bug report to IBM might be useful too.

MK