topic Re: memory allocation for programs in Operating System - Tru64 Unix

memory allocation for programs

Tom Wallin_1 — Wed, 19 Jan 2005 16:22:46 GMT

I am wondering how the loader part of an executable calculates the
amount of memory to set aside for each program a user runs in his
environment. That is when a user types in a programs name at the prompt
and hits "Enter" How does the loader figure out how much memory to set
aside for it to run. I think it needs some for code, some for stack
space, and some for data.

We have identical programs, running on different servers, that are
grabbing two different amounts of memory when they are run.
The two different servers are as follows -
1. Alpha 4100 running V4.0F PK7 - has 3GB RAM
2. Alpha ES40 running V5.1b PK4 - has 8GB RAM

On the ES40 the program grabs 10x the amount of memory that it does on
the 4100.

Tom Wallin

Re: memory allocation for programs

Hein van den Heuvel — Wed, 19 Jan 2005 21:34:11 GMT

What do you use to tell the size?
ps -o RSS,VSZ ?

RSS is Real memory (resident set)
VSZ is Process virtual address size ... the program may or might not touch that all.
I suspect that RSS is similar, VSZ different.

I would also compare output for sysconfig -q proc

And check: ulimit -a

Hope this help some,
Hein.

Re: memory allocation for programs

Ravi_8 — Thu, 20 Jan 2005 01:17:15 GMT

Hi

can you post output of "ulimit -a" on both machines

Re: memory allocation for programs

Michael Schulte zur Sur — Thu, 20 Jan 2005 04:23:49 GMT

Hi,

the two machines have a substantial difference in built in memory. There is a parameter that tells tru unix how much of a programme to hold in memory. With almost three times the memory size the es40 can hold more in memory.

greetings,

Michael

Re: memory allocation for programs

Tom Wallin_1 — Thu, 20 Jan 2005 09:46:01 GMT

We are using RSS to compare. Attached is a text file(MS) with the ulimit -a from both machines.

Regards,

Tom Wallin

Re: memory allocation for programs

Hein van den Heuvel — Thu, 20 Jan 2005 10:03:43 GMT

Your STACK size seems unreasonable:

ES40 ulimit -a
:
stack(kbytes) 4,194,304

That's 4 GB of stack, where you used 2MB before. The app worked with the 2MB right? So it shoudl still work with that.
4GB of stack is a whole bunch of pagetable pages also!
If you do not have a very good reason for to have set it that high, then just fix the sysconfig proc entry for the stack and be happy.

Hein.

Re: memory allocation for programs

Tom Wallin_1 — Thu, 20 Jan 2005 10:13:44 GMT

We decided yesterday to do as you suggested. But we are going to adjust the data setting also. Why did you focus in on the stack only? Also does the loader use these values to calculate how much memory to set aside when loaded into memory?

Tom Wallin

Re: memory allocation for programs

Hein van den Heuvel — Thu, 20 Jan 2005 11:47:23 GMT

>> Why did you focus in on the stack only?

a) Because I am in a BEA training with limited downtime :-).

b) Because that value is so clearly wrong (assuming you had a working system).
Typical sizes are 32M or so. If you need more (lots of local variables) then you wouls probably know.

c) I could imagine stack being different, as pages on either end will actually be touched.
'data' is just a fence. Just telling the system how far it can go, but you probably will not go there, so it does not matter if it is too far. However... if an application loses control, then it will not be stopped by this clean fence, but by some nasty resource depletion which my hurt other applications.

>> also does the loader use these values to calculate how much memory to set aside when loaded into memory.

Yes... but that is all 'virtual'. Just address ranges.

Re: memory allocation for programs

Tom Wallin_1 — Thu, 20 Jan 2005 11:56:10 GMT

We changed the setting - per_proc_stack_size and restarted the server and started one of the programs and it was the same saize as before the change. We recompiled the program and restarted it no change.

The ulimit list of stack size did not change either.

What kernel variable should I be changing?

Tom Wallin

ps Thanks for your time.

Re: memory allocation for programs

Michael Schulte zur Sur — Thu, 20 Jan 2005 11:58:59 GMT

Tom,

could you post the output of the ps as Hein first suggested from both machines?

thanks,

Michael

Re: memory allocation for programs

Tom Wallin_1 — Thu, 20 Jan 2005 12:26:00 GMT

Michael,
I tried the ps -o but the ps cmd did not like the RSS or VSZ. It gives the msg - "ps: unknown variable in format: RSS".

I was wrong about the stack size not changing in my previous msg. The size did change.

Re: memory allocation for programs

Hein van den Heuvel — Thu, 20 Jan 2005 12:31:22 GMT

Oh, RSS could be the column header for display purposes, and rss the option field.
Just use 'man ps' and '/RSS'

Hein.

Re: memory allocation for programs

Michael Schulte zur Sur — Thu, 20 Jan 2005 12:44:39 GMT

Hi,

try this:
ps -eo pid,rssize,vsize,command

greetings,

Michael

Re: memory allocation for programs

Tom Wallin_1 — Thu, 20 Jan 2005 12:57:37 GMT

I got the ps -O thing now. Attached is the ps cmd from each server as a MS text file.

Tom Wallin

Re: memory allocation for programs

Tom Wallin_1 — Mon, 24 Jan 2005 10:42:33 GMT

I have tried tha things suggested. We are still stuck with the same problem. I have moved the question to the "General" part of the forum.

Tom Wallin

Re: memory allocation for programs

Tom Wallin_1 — Mon, 28 Feb 2005 11:07:51 GMT

I wanted to update this thread with the final fixes. Both were settings in sysconfigtab. One in the vm: section - vm_segmentation and one in the rt: section aio_task_max_num.
We started with a program that was taking 10MB of RSS per user. It turns out that the program itself takes 1.8MB of RSS. Another 3.6MB was taken when vm_segmentation was set to 1. This setting causes all libraries referenced by the program to be loaded with it into memory. This was for security purposes.
And the last amount of RSS 4.6MB was taken because we had the aio_task_max_num set to 60108. This setting causes the initialization routines in libaio to allocate 60108 aio result blocks. These are used to contain context for outstanding aio requests. At 80 bytes per result block the resulting memory allocation is 587 pages (4.6 Mbytes).
We think that the 60108 was set for Database purposes. But we are unsure why. We did not need the setting for Informix.

Tom Wallin