Pthread linking and libc stubs

[sharanrao]

Hi,

When we profiled our application using caliper, we found that a lot
of time was being spent acquiring locks whenever stl was being used.
Searching on the internet we found that stl automatically becomes
thread safe on linking with pthread libraries.
Our application links with thread safe libraries which are compiled
with libpthread. One of HP's sites
(http://docs.hp.com/en/1896/pthreads.html) + many articles written by
HP on libpthread, suggest resolving the pthread lock calls with the
libc stubs.

So, we did not build our "executable" with "-mt". Hence the load order
changed, and the calls to the pthread locks were resolved using the
stubs in libc.

Now, my question is - is this safe ? One of our experts had the following fear

" once we compile part of the code with “-mt” and part without it
creates a situation of inconsistency as variables for code compiled
with “-mt” contain additional data structures such as reference count
and others which is not available for code not compiled with “-mt”."

We link with many third party libraries like oracle client libraries,
which link with pthread.

Can you please let us know what type of issues can exist with the
approach we used ? Is it safe ?
Or can you refer us to some definitive document which will clarify the
pthread linking issues comprehensively.

Thanks,
Sharan Rao

[kobylka]

Hello Sharan!


Well, I'll try my best to answer your question...

Let's separate your post into three key aspects:

1. Threaded application that uses thread-safe libraries (normal case)

2. Non-threaded application that uses thread-safe libraries (your case)

3. Compilation and linking in threaded applications


Thread safe libraries need symbols from libpthread mainly for synchronization (BUT not exclusively!).

1. When linking threaded applications this synchronization is needed (in app and in libs) and hence libpthread is needed.

2. When linking non-threaded applications this synchronization is not needed (neither in the app nor in the libs, but libs need to resolve their references) and hence HP allows you to link against the dummy pthread* functions in libc (see link you provided). This is a way to speed up execution in libs (thus overall execution of your app).

In case 2. there will be no problems as long as your application does NOT make use of a library that needs pthread* functions for more than synchronization (for example to create threads). This library/ies will have its references resolved with the stub functions in libc too and therefore will stop working correctly.


3. So now we can compile and link a threaded program:

- using only -mt
- using only -lpthread
- using only -lc (dummy pthread functions).

If only using -lc, case 2. applies to your app.

The difference between using -mt COMPILER flag and -lpthread LINKER flag is that the former adds preprocessor defines that are used during compilation (and also the -lpthread flag to the linker), so the safe (and correct) way to compile threaded apps on HP-UX is using -mt!


Now back to your case, if you are sure the libs you are linking your non-threaded app to will be happy with the dummy pthread* functions in the libc stub, then no prob. If not or if unsure do not risk, link to libpthread (using -mt ;).

Hope this helps.


Kind regards,

Kobylka

[Dennis Handly · Accepted solution]

>is this safe?

Sure, if you don't create threads, there is no need to link with libpthread.

>"once we compile part of the code with "-mt" and part without it creates a situation of inconsistency as variables for code compiled
with "-mt" contain additional data structures such as reference count and others which is not available for code not compiled with "-mt"."

No, we very cleverly make the structs the exact same size, whether -mt or not. So this will work, if you don't link with libpthread.

>We link with many third party libraries like oracle client libraries, which link with pthread.

Then you are hosed. You need to use chatr(1) and LD_LIBRARY_PATH to point to an "empty" shlib named libpthread.so.1.

touch empty.c
cc -b -o libpthread.so.1 empty.c

>Or can you refer us to some definitive document which will clarify the pthread linking issues comprehensively.

http://docs.hp.com/en/14487/threads.htm

>kobylka: I'll try my best to answer your question.

You did very well.

[rApt0r]

If my libraries expects linking with libpthread but my end application is not going to run multiple threads, then  is there e major performance benefit when comparing linking native libpthread vs empty library ( using stub calls in libc ).??

[Dennis Handly]

>If my libraries expects linking with libpthread but my end application is not going to run multiple threads

If your shlibs expect you to link your executable against libpthread, then you should be find if you just use the libc stubs.  And you shouldn't have to use that kludge for that empty libpthread.

>is there a major performance benefit when comparing linking native libpthread vs empty library??

Yes.  Especially if you don't get a benefit of using multiple threads.