memory allocation in unix

[Tom Wallin_1]

Does anyone know what system attributes contol how much memory a program is given. I know this is a really generic question with many possible answers. But I have programs that are running on V5.1b of Tru64 Unix that give the programs a certain amount of memory. The same programs on another machine with the same version of Tru64 unix gives the programs 10X the amount of memory. With 450+ users and rising we are running out of RAM memory and the system is paging and swapping.

Tom Wallin

[Ross Minkov]

Check out the System Configuration and Tuning guide. It's available online at http://h30097.www3.hp.com/docs/base_doc/DOCUMENTATION/V51B_HTML/ARH9GCTE/TITLE.HTM

Regards,
Ross

[Ravi_8]

Hi Tom

Allocation of memeory to a program depend on various factors, like physical memory existing in the machine, swap, shared memory,semaphores existing in the machine. It might be diffrenet in 2 of your machines.

Hence all the standard softwares(oracle,db2,websphere, etc..)need defined values set for them. They will be in README (or documentation) of that particular software.

[Ralf Puchner]

I see you have never written a program ;-)

If adding more memory to a machine the kernel variables will be calculated and used on the current memory configuration. Have a look into the tuning guide explaining the formulars and default values.

Depending on the kernel settings the proc, data memory will be bigger, the memory allocation of the program depend on the used method memory is allocated (malloc, shared memory etc.). Have a look to the limit/ulimit command man-page.

With "ps waux" you will get the virtual/real memory size of the program (shared memory not included). Shared memory can be monitored by using ipcs.

The used allocation method for the virtual-memory is also an issue. Because in default swap mode the os will reserve memory so memory is limited by the swapspace not by physical memory. But this is also well documented within the admin guide - maybe a starter here to get the basics of memory allocation...

[Tom Wallin_1]

I have not written a program in unix. But I have written programs in a real time multitasking OS so I do understand the mechanics of memory allocation.
And yes I could read manuals until I happen upon the explanation of Tru64 UNIX memory allocation. But that approach is taking too long.
I am reading the Configuration and Tuning manual for 5.1b. And I am in contact with our database support.
We have been looking at the memory usage with ps and vmstat. The issue is still the amount of memory that the OS is giving a program.
Ralf, What do you mean by - used memory allocation. Please explain to the novice. :-)
And which section of the System Administration manual has the "basics of memory allocation.

Thanks,
Tom Wallin

[Ross Minkov]

Tom,

I couldn't help noticing that you have 2 identities in ITRC. You used one to open this thread and just now you used the other one to post a follow-up. I assume it might be OK to have more than one ITRC identity, but you should be using the same identity you used to open a thread when posting follow-ups.

Check this book -- Tru64 UNIX-Oracle9i Cluster Quick Reference

Here is a link:
http://www.amazon.com/exec/obidos/tg/detail/-/1555582729/qid=1106585284/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/102-9914536-8925741?v=glance&s=books&n=507846

In case the link doesn't work just go to amazon and search tru64 + oracle

Regards,
Ross

[Tom Wallin_1]

I am sorry about the dual id. I am going to fix that as soon as possible.
What is it that I should be looking for in the Oracle link document?

Tom Wallin

[Ralf Puchner]

used memory allocation method means how the program handles memory, so what method (api/functions) will be used? Consider also the calculation of the stack/heap, reserved space etc. which is done by kernel parameters and calculated on the configuration (e.g. if starting a program allocates 8MB on one and 2MB on another system depending on the configuration of the machine) - so thats the reason why programming knowledge is a starter here ;-)

Depending on the used api/functions you can decide or check the necessary kernel parameters. But this is identical to all OS, not unix at all ;-).

In case of Oracle there is a tuning guide which explain what parameter and which formula must be used to calculate the correct values for your configuration. But this documentation should be read prior to install the database itself.

[Tom Wallin_1]

>> Consider also the calculation of the stack/heap, reserved space etc. which is done by kernel parameters and calculated on the configuration (e.g. if starting a program allocates 8MB on one and 2MB on another system depending on the configuration of the machine) - so thats the reason why programming knowledge is a starter here ;-) <<

This is what I am asking help on. As I stated I have programmed in the past. But it has been a few years. Yes I can chase down documentation to try and find the answer to the question, that you so eloquently stated above, but I have a production system that needs a fix now. And I have other duties that are getting neglected while I try and relearn programming.

>> Consider also the calculation of the stack/heap, reserved space etc. which is done by kernel parameters and calculated on the configuration <<
What is the formula? :-)
Tom Wallin

[Tom Wallin_1]

We found out today that on systems with C2 security enabled the vm section of sysconfigtab has a parameter called vm_segmentation. The parm either allows global access to libraries or loads the libraries in with a program when started. Setting vm_segmentation = 1 is for global access and =0 is for loading the libraries in with the program. We gained 3.7MB back, with RSS, from each loading of the program that we were watching. With 450 users we are going to get back around 200K memory pages. One strange thing was that the VSZ remained the same size.

:-)
Tom Wallin

[Ralf Puchner]

Tom,

the formula is within the admin guide, please go through it (on cd or here on the webserver), click to admin guide, kernel parameter or simple use the man or apropos command. It is really difficult and needs a lot of time for that action ;-)

And if this is a production system why still wasting time for days of an answer here in the forum? It is faster to call the HP support center - but this is only a hint ;-)

[Tom Wallin_1]

I wanted to update this thread with the final fixes. We got the answer from HP support last Friday on the sysconfigtab setting aio_task_max_num. The vm_segmentation setting was given to us around Feb 15th.
Both settings are in sysconfigtab. One in the vm: section - vm_segmentation and one in the rt: section aio_task_max_num.
We started with a program that was taking 10MB of RSS per user. It turns out that the program itself takes 1.8MB of RSS. Another 3.6MB was taken when vm_segmentation was set to 0. This setting causes all libraries referenced by the program to be loaded with it into memory. This was for security purposes.
And the last amount of RSS 4.6MB was taken because we had the aio_task_max_num set to 60108. This setting causes the initialization routines in libaio to allocate 60108 aio result blocks. These are used to contain context for outstanding aio requests. At 80 bytes per result block the resulting memory allocation is 587 pages (4.6 Mbytes). This memory was set into wired memory.
We think that the 60108 was set for Database purposes. But we are unsure why. We did not need the setting for Informix.

Tom Wallin