Why so many threads?

[Jose M. del Rio]

Hi,
we are using a 3rd-party executable. After upgrading to its latest version, we notice it allocates now a lot of memory, just since the beginning.
By using tusc and glance we can see it is creating on startup as many threads as processors there are, each with a 8 Mb mmap() region. The previous version created only one.
According to its developers, they are not creating the threads expressly nor compiling the program with the +Oparallel switch but the matter is: setting the environment variable MP_NUMBER_OF_THREADS=1 does the trick and only one thread is created.
Questions:
1) How can this be? Is there another compiler option that can lead to the creation of parallel threads?
2) Given an executable, is there a way to know the options it was compiled with?
Thanks.
Regards.

[Steven E. Protter]

Shalom,

You don't say what the third party executable is.

You updated the program and its performance characteristics changed.

You did not patch your HP-UX system or make changes to it.

The developers are not accurate, they changed something.

Most of the time when something like this occurs its a result of the last thing you did, which was to update the executable.

It is possible you did so without following system prepartion guidelines, but you should have been notified of that, certainly when you contacted the application vendor.

If you mentioned the name of the third party product, you might find someone who has experienced this issue can provide assistance. Right now you've given us very little to go by.

SEP

[Emil Velez]

ON HPUX 11.0 and later the OS supported kernel threads. A programmer creates multiple threads to do parallel operations on different processors. Evidently the programmer put a capability to limit the number of threads which limits the programs capability to utilize a multiprocessor box. Threads are a valid way of modeling applications so a multithreaded application is not a problem. In fact some of the HP-UX OS applications like the midaemon and vxfsd and other processes have multiple threads. IF you want to see this if you have glance you can goto the process report and add the column "PROC_THREAD_COUNT" to the report and see what processes on HP-UX utilize multiple threads.

[Dennis Handly · Accepted solution]

>1) How can this be? Is there another compiler option that can lead to the creation of parallel threads?

+Oopenmp will also create threads.

>2) Given an executable, is there a way to know the options it was compiled with?

If you have a recent linker patch, use /usr/ccs/bin/footprints.

You can also use chatr(1) on the executable and look for libcps.

[Jose M. del Rio]

Steven: the executable is 'gsrvr.shared'. It is included in ESRI's ArCSDE software.

Emil: in this case, the problem with having multiple threads is: a gsrvr is a dedicated server process, servicing a unique client connection. Our server has 16 CPUs so every gsrvr creates on startup +15 additional threads. Each thread has, among other regions, a 8 Mb MMAP region. That makes 15*8 Mb = 120 Mb per gsrvr. It is not uncommon for us to have 400 concurrent connections => 400 * 120 Mb = 48 Gb. Our swap space gets exhausted.

Dennis:
- chatr shows it was linked with libcps.2. Does this mean either +Oparallel or +Oopenmp were used?
- none of our servers have the footprints command. Do you know which product we should install?

Thanks to all of you.

[James R. Ferguson]

Hi Jose:

For 11.23 PHSS_34860 would bring you the 'footprint' binary. It's contained in the lastest linker patch as Dennis noted.

For 11.11 use patch PHSS_35381.

If you are running 11.31 you should already have it. The online manpages for 11.31 offer insight into its output.

Regards!

...JRF...

[Dennis Handly]

>- chatr shows it was linked with libcps.2. Does this mean either +Oparallel or +Oopenmp were used?

Yes.

[Jose M. del Rio]

Thanks James & Dennis.
It's incredible how useful this forum can be :-)
Here is footprint's output.
According to the man page, I guess the fact that it can't find any compilation unit is because the program was not compiled with an HP compiler? Or maybe some compiler switch disabled footprint in the object file?
It seems elfdump can't find compilation units either.

ensnada6:/arcsde/sdeexe91/bin#/usr/ccs/bin/footprints -v gsrvr.shared
Scanning gsrvr.shared ...
Shared library: /usr/lib/pa20_64/libCsup.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libstream.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libcl.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libnsl.1 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libdl.1 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libm.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/librt.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libF90.1 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libcps.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libpthread.1 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libnss_dns.1 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libstd.2 (system library - not scanned)
Shared library: /usr/lib/pa20_64/libc.2 (system library - not scanned)

Compiler footprint summary

Number of compilation units: 0

Earliest compiler timestamp: N/A
Latest compiler timestamp: N/A
Linker timestamp: N/A

Total size: 520.9K (text) 19.3K (data)


ensnada6:/arcsde/sdeexe91/bin#elfdump -dc gsrvr.shared

gsrvr.shared:


Compilation Unit Dictionary from gsrvr.shared:


Language Name
Product_id ; Version_id
Compile time; Source time

[not present]

Regards.

[Dennis Handly]

>I guess the fact that it can't find any compilation unit is because the program was not compiled with an HP compiler? Or maybe some compiler switch disabled footprint in the object file?

The executable has probably been stripped. Since you do have libCsup, it was compiled with aC++.

[Dennis Handly]

>The executable has probably been stripped.

Hmm, my strip won't work on ELF files. While I've filed a bug report on that, I wasn't aware it was fixed yet.