HPUX 11.00 Dynamic Library Build Problem

[Fred Vogel]

Hello All,

I have successfully built our software on an 11.23i machine (32bit build) with no problem When I go to build the same load on my 11.0 PA system the code builds and links, but at run time the dynamic linker blows up with the error "Exec Format Error." when trying to load my library.

ldd -r reports the following unresolved symbols in my .sl build:
symbol not found: _SYSTEM_ID (/usr/lib/libc.2)
symbol not found: _end (/user/lib/libc.2)
symbol not found: __dld_loc (/usr/lib/libdld.2)
I do not have issues with unresolved symbols on the 11.23i machine for this build.

I have set the following build options:
Compiler: +DAportable -WL,+s -WL,-E, -z -D_HPUX_SOURCE -D_HPUX_10_20 -Aa -ext +z(PIC) -c(create .o files)
Linker: -b(build shared library) -E(export all symbols) +s(use shlib_path)

In an attempt to resolve the problem I have tried to run nm on all of the .sl's in /usr/lib but cannot find these symbols defined in anything. I have also tried to make sure that I have patches installed that provide updates that have fixed unresolved symbols in the hp provided libraries. Although, I don't have any idea if I have installed them all.

On the 11.23i build I am still using shl_load instead of dlopen; so no code has changed only the build environment.

Any help would be appriciated. I will be happy to provide data anyone might need to assist. I am still somewhat new to the HPUX world, so for quicker turnaround on response from me, assume that I do not know how to gater the information you want and let me know the command so that I can provide the right data as soon as possible.

Thank You
Fred

[Steven E. Protter]

Shalom Fred,

Looks like a code change is required.

There is no guarantee that code that works on 11.23 will be backwardly compatible to earlier releases.

SEP

[Fred Vogel]

Thank you for your reply; however, the software was written on 10.20 and then moved to 11.0 and then to 11.23. At one point these libraries built and ran on the 11.0 box and now they don't.

I am new to my position here and don't know the history of the box. It is a development box and is primarily maintained by developers and not IT. Since I can't find the unresolved symbols, i suspect the answer is either a patch to fix an incomplete library or a patch removal to unbreak a library.

I provided the additional into becase it is possible that I do not have all the information I need and a tweak to something i'm doing could be the fix.

[Fred Vogel]

Left a question out. Since the unresolved symbols are in default libraries, what change could I make to my code that would effect that?

Thanks
Fred

[Dennis Handly]

(There is no such thing as "11.23i". Did you mean IPF?)

>at run time the dynamic linker blows up with the error "Exec Format Error." when trying to load my library.

Is it the right type? What does file(1) say?
Other issue could be due to the shared lib high water mark.

>ldd -r reports the following unresolved symbols in my .sl build:
_SYSTEM_ID _end __dld_loc

This is a useless command to use on shlibs. What are you looking for?

>Since the unresolved symbols are in default libraries, what change could I make to my code that would effect that?

The unsats come from system libs and are defined in the a.out.

>-WL,+s -WL,-E,

I assume you meant "-Wl". And there shouldn't be a "," after the "E".

Are you sure you aren't linking with a shlib that was created on 11.11 and you are trying to load it on 11.00? That may get that "Exec Format Error".

If the output from file(1) isn't obvious, we are going to have to go back and forth on the "high water mark" issue.

[Fred Vogel]

Hello Dennis,

Thank you for your reply. I will try to answer your questions:

What does file say?:
libMYpexa.so: PA-RISC1.1 shared library -not stripped.

ldd is a useless command:
Is there an alternative tool that will report unsats at dynamic link time?

Incorrect inputs -WL,+s -WL-E,:
I looked in the build files, these were actually typos from when I created my message. In the build files they are actually -Wl,+s -Wl,-E

Using a shlib created on 11.11?:
This is a good question. I can tell you we have never had an 11.11 machine so my shlibs are not 11.11 versions; however, I guess it would be possible that something in /usr/lib could be from 11.11. How can I query the files to find out?

High Watermark:
I don't know what this is. If you need some metric from the system, let me know and I will provide it.

Thank you for your help.

Fred

[Fred Vogel]

Just to follow up on the 11.11 question.

I have looked through swlist at all of the installed software, patches, and bundles. Everything says that it is for 11.00 architecture. Based upon this I assume that means all of my default libraries are for 11.0.

Thanks Fred

[Dennis Handly]

>Is there an alternative tool that will report unsats at dynamic link time?

You can use show_remaining_imports, attached.
It uses vector methods with elementary text processing to compute it.
This tool was distributed with the obsolete cfront.

>How can I query the files to find out?

There is no good way, if you have the same compiler version on both 11.00 and 11.11.

>High Watermark:
>I don't know what this is. If you need some metric from the system, let me know

I'm going to to have to reverse engineer it, for the Nth time. ;-)
Before we go there, was there anything else besides "Exec Format Error."??

This error would occur if you built on 10.20 and took it back to 10.10.

Though last year I helped someone that got this while trying to stub out some Oracle shlibs.

To track any high water mark issues you need to use /usr/ccs/bin/odump:
$ odump -slliblist -sllibload /opt/aCC/bin/aCC
Shared Library List Table for /opt/aCC/bin/aCC:
Index Ref IDNRVIL HighWater Name
0 abs .D..... 0 aCC.new
1 -l .D...I. 0 /usr/lib/libm.2
2 -l .D...I. 49 /usr/lib/libcl.2
3 -l .D...I. 0 /usr/lib/libc.2
4 abs .D...I. 0 /usr/lib/libdld.2

Shared Library Load List for /opt/aCC/bin/aCC:
Order Name
0 /opt/aCC/bin/aCC
1 ^ /usr/lib/libm.2
2 ^ /usr/lib/libcl.2
3 ^ ^ /usr/lib/libdld.2
4 ^ ^ /usr/lib/libisamstub.1
5 ^ /usr/lib/libc.2
6 ^ ^ /usr/lib/libdld.2
7 ^ /usr/lib/libdld.2

Notice the 49 for libcl.2

To see what is recorded in the shlib:
$ odump -sldlhead /usr/lib/libcl.2
Shared Library DL Header for /usr/lib/libcl.2:
version: 93092112
LTptr_offset: 0x00000bc0
highwater_mark: 0000000049 <<<
...

(Of course you would need to use these commands on your shlibs. And all your dependent shlibs.)

You would get this error if aCC was run against an older version of libcl.2 that had a smaller high water mark than 49.

A quick check of my 11.00 system shows that only libcl has a high water mark, an indication of that evil intra shlib versioning: :-(
http://docs.hp.com/en/7133/pragmas.htm#pragma-SHLIB-VERSION

[Fred Vogel]

Hello Dennis,

I apologize for not getting back with you sooner, I had easter plans and did not get your reply until this morning. I do not know where you are, I am in US Central time.

First to answer your question: The only reported information is the "invalid exec format" output I mentioned earlier. Looking at ERRNO it is 2; which indicates no such file or directory. I followed this in the WDB/GDB and found that the input path and names are exactly correct and the place that it is crapping out is in shl_load. (as would be expected for the exec format error...or so i think.)

Thank you for all the information. I DL'ed your attachment and it works.

I followed your odump examples and got the exact same results. I will see what the output is for my executibles and shlibs. As soon as I have results I will post them. I also still have to go read the documentation you pointed me too.

Thanks
Fred

[Fred Vogel]

Ok, I solved one problem that was a stupid typo. I kept looking over my stuff and was concerned about ERRNO being 2.

That turned out to be a trailing '/' in the library search path for my oracle libs. This was revealed by chatr. Now that error is gone...BUT the "exec format error" persists. ERRNO now reports 10 - No Children, which I don't quite understand.

Fred

[Dennis Handly]

>That turned out to be a trailing '/' in the library search path

While not elegant, this shouldn't matter.

> I also still have to go read the documentation you pointed me too.

I don't care if you read it, just don't use it!

[Fred Vogel]

Hello Dennis,

I've been out of the office for a few days with an ill family. Everything is better now.

So, I found something interesting. I checked to make sure we were not using the pragma you mentioned...and it is not used anywhere.

I then proceeded to perform an odump on all of the libraries that were to be loaded to see what the highwater marks were. What I got was odump complaining that several libraries in /usr/lib had unexpected header sizes and warned that this was likely due to an outdated dld.sl, ld, or odump. I think that is probably true of all three.

Do you think this is actually the source of all my build problems with shlibs on my 11.0 system? I have looked around for patches or updates for the ld/dld software but can't find it. I sent an email into hp asking where I can get support for this issue. Do you happen to know where i need to look for these updates?

Thanks,
Fred

[Dennis Handly]

>I checked to make sure we were not using the pragma you mentioned...and it is not used anywhere.

I didn't think so. But the use in libcl could cause problems with other shlibs that have dependencies on it.

>What I got was odump complaining that several libraries in /usr/lib had unexpected header sizes and warned that this was likely due to an outdated dld.sl, ld, or odump.

>Do you think this is actually the source of all my build problems with shlibs on my 11.0 system?

I wouldn't worry about it.

>I have looked around for patches or updates for the ld/dld software but can't find it.

The last patch is PHSS_35380. It does have a warning on it, so you may want a previous version.

>I sent an email into hp asking where I can get support for this issue.

11.00 is no longer supported.

[Fred Vogel]

Dennis,

I thought for sure that would be the problem and my fix was only a patch away. I didn't see the warning on PHSS_35380. I checked my swlist and installed it last week.

I did run odump on all libraries that would get loaded by the shlib in question and the high water marks were almost all 0 with 2-3 being 49.

I have attached the odump results for your review.

Thank You as always!

Fred

[Dennis Handly]

>I thought for sure that would be the problem and my fix was only a patch away.

No, your problem is that you are linking "incorrectly".

>I did run odump on all libraries that would get loaded by the shlib in question and the high water marks were almost all 0 with 2-3 being 49.

You need to do odump -sldlhead on your executable, and any library that had 49:
4 -l .D...I. 49 /oracle/u01/app/oracle/product/8.1.7/JRE/lib/PA_RISC/native_threads/libjava.sl
12 -l .D...I. 49 /usr/lib/libcl.2

So the issue may be that libjava.sl??

The last time this happened, someone was trying to stub out something like libjava.sl with 49, with a dummy lib that had 0.

[Fred Vogel]

Dennis,

Attached is the dl header output you asked for. I also provided the lib list table and and load list for libclntsh.sl (linked as *.sl.8.0) and libjava.sl since I didn't remember if I provided those 2 before.

From what I can determine with my limited understanding of what I'm looking at, it looks like everything with a high watermark of 49 is because of libcl.2. (libcl.sl -> libcl.2)

Thanks,
Fred

[Fred Vogel]

Bump

[Dennis Handly]

>From what I can determine with my limited understanding of what I'm looking at, it looks like everything with a high watermark of 49 is because of libcl.2.

Yes. We had the opportunity to remove that and reset back to 0 for 11.0 but we didn't realize it in time. That would have prevented a few issues. :-(

Sorry for the delay. I looked at them and I don't remember what the smoking gun looked like nor do I see it in your data.

You don't have two libs with the same basename but different paths and you use SHLIB_PATH or differing shl_loads to select one or the other??

I suppose you could just link your executable with -lcl, and the highwater mark issue would be moot.

In looking at all dld errors that have "Exec Format Error", I see some come from munmap.

So could you use tusc to trace the execution?
$ tusc -tp -o tusc.out your-executable ...

If anything, the tusc output may point to the lib in question, see below.

Back to the original error, this is what I'm looking for:
/* if the library on the system is older than version recorded at link
time then give an error */
if(dl_hdr->highwater_mark < min_hwm && min_hwm != NIL) {
/* unmap text */
munmap((caddr_t)taddr, tsize);
errno = ENOEXEC;
return(INVALID_VERSION);

Which should have this error:
Invalid version for shared library:

Perhaps if you changed your shl_load parms to OR in BIND_VERBOSE, it would would show the error message?

[Fred Vogel]

>Sorry for the delay. I looked at them and I don't remember what the smoking gun looked like nor do I see it in your data.

Don't worry about the delay, I'm just greatful for your help.

>You don't have two libs with the same basename but different paths and you use SHLIB_PATH or differing shl_loads to select one or the other??

No, I checked and double checked that.

> I suppose you could just link your executable with -lcl, and the highwater mark issue would be moot.

I could do that, which wouldn't be a big deal if it were just for one exe; but I would have to integrate this library into every exe that used it. I will give it a try and see if I can get it to work or if it gives me some other problem.

> So could you use tusc to trace the execution?

I havent been able to find tusc for this machine; the one I have for my 11.23 box is incompatible. I found an 11.11 version but am leary of putting a different OS version's software on the machine. Is there an 11.0 version of tusc?

>Perhaps if you changed your shl_load parms to OR in BIND_VERBOSE, it would would show the error message?

I will give this a try today and post the results.

Thank you
Fred

[Dennis Handly]

>I found an 11.11 version but am leary of putting a different OS version's software on the machine. Is there an 11.0 version of tusc?

I assume there was a 11.00 version. But if you can't find one, try the 11.11 one.

[Fred Vogel]

Dennis,

I was able to find an 11.0 version of tusc and it works well. I am sorry I have not replied with any additional information.

I have moved to work on some software enhancements on another platform for the time being. So I can't pursue this any more for now.

I want to thank you very much for sticking with me and trying to help. Ultimatly we are going to have to resolve this issue, so at some point I will probably bring this back up.

I will finish assigning you points for the time you have invested in trying to help me solve this problem.

Thanks,
Fred