HP-UX 11i memory utilization question

[Tom Wolf_3]

Hello all. I have an N4000 server running HP-UX 11i. This server has eight 550 MHz PA 8600 CPU's and 16 GB of physical memory. According to SAM, about 12.5 GB of total virtual memory is available.

Now comes my question, an application analyst posed the following questions to me concerning what he saw in glance for a particular process:

What is the virtual memory limit for a single threaded process?

If there is a limit, what impact does reaching it have on the process?

What is the difference between resident memory (RSS) and virtual memory (VSS)?

I believe I may know the answer for a few of these questions but would rather get some confirmation from the experienced individuals who respond to these forum questions.

As always, any assistance is greatly appreicated.

Thank you.

[Jeff Schussele]

Hi Tom,

Well...there are several limits as follows:

maxdsiz # max data seg size
maxssiz # max stack size
maxtsiz # max text size

Caution needs to be taken so that none of these are set *so* high as to "hamstring" the server such that it comes to grinding halt.

And it should be noted that you append "_64bit" to all these for that address space.

HTH,
Jeff

[A. Clay Stephenson · Accepted solution]

The answer varies a bit with the kind of executable. 32-bit processes are limited to a 4GB virtual address space; 64-bit processes (and threads) for our purposes essentially have no practical limits in that the machine will run out of virtual memory long before you reach the limits. There are also limits imposed by kernel tunables like maxdsiz and maxdsiz_64bit which will likely come into play before all the machine resources are exhausted. Virtual memory is the sum of physical memory and swap space and can thus be much, much larger than physical memory.

VSS is the total size of a process image which RSS is the size of the image actually in physical memory at that moment. Note that a demand-paged OS may have parts of a process swapped out at any given moment.
You should also note that trying to calculate system-wide process memory totals is far from trivial. For example, consider 500 vi processes; each would have a private data segment and a private stack but would share a common text segment (the instructions) as well as shared library code. Do these common parts get counted 1 time, 500 times, or something in between?

[James R. Ferguson]

Hi Tom:

Processes have limits for code (text), data and stack sizes governed by kernel fences (limits) 'maxtsiz' 'maxdsiz', 'maxssiz' respectively. 32-bit processes use one set of bounds while 64-bit ones use another.

A process's memory footprint is spoken of as the includes all three of these components.

Part of the problem in measuring this footprint size is whether or not you include shared memory segements (which another process uses with yours) and/or the memory needed for shared libraries called from your process.

Glance's RSS includes shared memory. The VSS value is more an estimation without this if I recall correctly.

Regards!

...JRF...

[Tom Wolf_3]

Thanks for the responses. I have follow up questions.

What if a process dies abnormally, is HP-UX smart enough to de-allocate any virtual memory that was being used by that process?

What's a good way to check for memory leaks associated with a given process?

[James R. Ferguson]

Hi Tom:

While memory is cleaned-up for a process that dies, killing a process with 'kill -9' which cannot be trapped, *can* leave shared memory segments left around. This is why you avoid this ungraceful kind of kill except as a last resort.

You can test for memory leaks (which are *application* bugs anyway) by monitoring the virtual size of a process over time:

# UNIX95= ps -e -o "user,vsz,pid,ppid,args" | awk 'NR>1' | sort -rnk2

Note the blank (space) character after the equal sign and before the 'ps' command. This kimits the setting of UNIX95 to this command only.

Note, that the virtual size (vsz) is in kilobyte size pages (see the 'ps' man pages) and excludes buffer cache and other shared components such as shared libraries and shared memory regions.

A leaking process will grow over time.

Remember, however, that if a program calls 'free()' to free memoary it has allocated, that that memory is only freed for the program's use during its remaining life. That is, it is given back to the system until such time as the process actually terminates.

Regards!

...JRF...

[Tom Wolf_3]

Thanks again to everyone for the quick and detailed responses.