topic Re: memory management in Operating System - HP-UX

memory management

Elan Viswanathan — Fri, 08 Jun 2001 00:54:12 GMT

Greetings to everybody,

Here is J5000 [10.20] with 3G memory installed. It is runing mathematical simulator/solver.
---------------------------------------------------
# dmesg | grep Physical
Physical: 3145728 Kbytes, lockable: 2310124 Kbytes, available: 2651032 Kbytes

# top
...
Memory: 910748K (899208K) real, 959264K (946304K) virtual, 1161884K free Page# 1/5

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
1 pty/ttyp2 19244 user 255 39 925M 872M run 500:22 100.04 99.86 dessis

...
---------------------------------------------------
Two questions:

1. Top show 967736K real + 1134056K free only.
Is there any way I can release more memory for process?
[strangely, glanceplus shows account for 3gb !!
Phy 3.0g Sys mem:307m, Buffer cache: 368m
User mem:1.23g, Free mem:1.1g ]

2. In the above top output, the resident size of "dessis" is 925M.
It can take more than that. It did so in other 11.0 machine.
What kernal parameter(s) do I need to tweak to allow a single
process consume more, if not "all" available, memory?

TIA.
-elan

Re: memory management

A. Clay Stephenson — Fri, 08 Jun 2001 01:16:46 GMT

Hi,

I've run similar applications and yes you need all the memory you can get.

You should increase maxdsiz, possibly maxssiz,
and sometimes maxtsiz needs to be increased as well. If the program will run, maxtsiz is probably big enough. You might also need to increase shmmax. I would also make sure that I have enough swap and/or enable pseudo-swap (swapmem_on=1). Typically, disk i/o is not all that important so I would reduce dbc_max_pct to no more than 10% or set bufpages to limit buffer cache to no more than 100MB. This program might also benefit from using "memory windows" since you are running 10.20. You can search this site for that concept.

Happy number crunching, Clay

Re: memory management

James R. Ferguson — Fri, 08 Jun 2001 01:20:17 GMT

Hi:

From the Configurable Kernel Parameters (HP-UX 11.0) document:

http://docs.hp.com/hpux/onlinedocs/os/KCparams.OverviewAll.html

> The kernel parameters 'maxdsiz' and 'maxdsiz_64bit' specify the maximum data segment size, in bytes, for an executing process.

> 'maxssiz' and 'maxssiz_64bit' define, for 32-bit and 64-bit processors respectively, the maximum size of the dynamic storage segment (DSS), also called the user-stack segment, or an executing process's run-time stack. This segment contains stack and register storage space, and such.

> 'maxtsiz' and 'maxtsiz_64bit' define, for 32-bit and 64-bit processors respectively, the maximum size of the shared text segment (program storage space) of an executing process.

...JRF...

Re: memory management

Elan Viswanathan — Fri, 08 Jun 2001 01:29:46 GMT

Thanks guys. I will try your suggestions and come back here with results.

My current parameters are:

maxdsize 2063835136
maxssize 83570688
maxtsize 1073741824

dbc_max_pct 50
dbc_min_pct 5

Re: memory management

Elan Viswanathan — Fri, 08 Jun 2001 22:05:05 GMT

I could rise maxssiz, and shmmax only. Rst are all resported at teir maximum, even after applying patch PHCO_22268.
maxdsize 2063835136
maxssize 401604608
maxtsize 1073741824
shmmax 2063835136
By reducing the percent of dynamic buffer (dbc_max_pct), now more memory is shown free. Yet the resident size of the process does not go beyond 974M as reported in top. While the same program takes more than 1.7 G on other 11.x machine.

Any clue about grabbing all the memory for this process?

Re: memory management

Bill Hassell — Sun, 10 Jun 2001 02:18:06 GMT

A standard 32-bit program cannot address more than about 980 megs without special handling. In 11.0 64-bit version, there is a new fence called maxdsiz_64 which allows many gigabytes of data space--but it has no effect on a 32 bit program which cannot go beyond the data quadrant (slightly under 1000 megs). 10.20 has nothing like this and al programs are subject to the 980 meg limit except as mentioned below.

The maxdsiz value can be set much higher than the maximum usable value in a program as it is simply a fence to prevent problem prgrams from running wild. Thus, setting it to 2000 megs will have no effect (as you have seen).

However, you can link two quadrants together to allow up to 1750 megs for a 32-bit program. Look at the EXEC_MAGIC features mentioned in the memory managment while paper in /usr/share/doc. It's important to note that from a memory management point of view, 10.20 is very different from 11.0 64-bit. Note that the dynamic buffer cache should never be 50% max...perhaps 15-25% at most. But even at 50%, it will shrink automatically if processes need the space.