info about shared libs

[Satya_6]

Hi,

I have an application with around 150 shared libs and when I launch the application it takes around 500MB!!!!

of course it looks logical seeing the size of the shared libs. I was wondering what part of the shared lib constitutes the space in memory when loded. Is there a way to reduce the memory consumption on launch.

I must admit that its not very much possible to reduce the number of shared libs getting loaded at launch time.

Does combining the shared libs and reducing the number help?

The binaries are built on PA_RISC 11.11 and the code is mainly made of C and C++ compiled using A03.65 with +O3 optimization

TIA
Satya

[A. Clay Stephenson]

500MiB is really is really not considered that large these days although having to have 150 shared libraries is a bit of a pain from a maintenance/installation point of view. The entire shared library is not loaded --- only those functions which are actually used and even then, the idea for most modern flavors of UNIX, is to treat them as pafe faults and load them just in time.

I suspect that startup times can be quite long as well. One thing that might help is a combination of static (archive) libraries and shared libraries. That would not reduce the memory footprint but it would reduce the load times and make maintenance easier.

One thing that might help your situation is the use of shared memory and a separate initialization program. I've written a number of applications which took many tens of seconds to initialize and were quite large because each instance had to setup tons of data before actually starting to work on a given task. The fix for that class of applications is to have a separate initialization program that reads in the data and loads a shared memory segment and then exits. The "real" application then only has to attach this pre-configured segment and is ready to go. That technique turned a 3 minute startup into about 5 seconds and greatly reduced the footprint because only the initialization program needed the initialization code.

[Dennis Handly]

150 seems like too many shlibs for an application.
What does the sum of the values of size(1) show for the executable and all of the shlibs?
Is the 500 Mb all data+bss? If text, that's shared so don't worry. Data is multiplied by the number of users running the application.

>what part of the shared lib constitutes the space in memory when loaded. Is there a way to reduce the memory consumption on launch?

There is the text space that contains code and read only data. It also contains the shlib info. Then there is the data area, which includes dld info on where each symbol is located.

By compiling with +Olit=all or +ESlit may move RW data to RO data.

>Does combining the shared libs and reducing the number help?

Possibly. Each shlib area is rounded up to 4Kb pages or larger. You could also try linking with -Wl,+mergeseg.

>with +O3 optimization

That may increase the size of text so that's not a big issue.

>Clay: The entire shared library is not loaded

But the data part takes up swap space.

[Satya_6]

Hi,

Thanks for the pointes. I missed to inform you one thing, that my application is a 32bit one and if 500MB goes just to launch, then I am left with a couple of GBs (that after enabling the +q3p).

Sorry I didn't give the output of the size command in the first place

running size on all my sl files, tells that around 350 MB is the text size, 140 MB is the data and another 10MB of bss.
and currently the executable is enabled with +q3p option

I guess the +ESlit is the default and I doing a chatr +mergeseg enable on all the .sl didn't change the scenario

And using the shared memory for initialization of the app is interesting and I would see how I can make use of this idea.

Thanks again for the suggestions
Satya

[Dennis Handly]

>that my application is a 32bit one and if 500MB goes just to launch, then I am left with a couple of GBs
>running size on all my sl files, tells that around 350 MB is the text size, 140 MB is the data and another 10MB of bss. And currently the executable is enabled with +q3p option\

You don't need +q3p if you only need 140 + 10 Mb of data. Of course it depends on how much additional heap you use.

>I guess the +ESlit is the default

Not for C.

>I doing a chatr +mergeseg enable on all the .sl didn't change the scenario

That allows large pages to be used for performance. It could slow startup.

>And using the shared memory for initialization of the app is interesting

You can't use shared memory to store C++ objects with virtual tables.