memory questions

[Kenan Erdey]

Hi;

a time ago, a hp-ux 11.11 machine(pseudo swap on, 4 gb device swap, 8 gb ram) gived an error like "Deferred swap reservation failure" and processes were being killed. sys_mem was about 1.7 gb. So i changed dbc_max_pct from 50 to 15.
and i added another swap device. after restarting sys_mem decreased to 1.1 gb. but i couldn't find why sys_mem was so high.Oracle forms and reports service is running on this machine.

i have some questions that i can't answer myself.

1. is sys_mem that we see in glance consisted of the data structures for virtual memory mechanism and process management ? what kernel parameters effects this value (for example nfile is 65550 in this machine)


2. Why pseudo-swap is seen as used 938 mb ? 5332mb reserved swap can be supplied from device swap before pseudo-swap.so no physical memory is used.


Swap Device Type Avail Used Priority
--------------------------------------------------------------------------------
/dev/vg00/lvol2 device 4.0gb 0mb 1
/dev/vg00/lvswap device 8.0gb 0mb 2
pseudo-swap memory 6.1gb 938mb -1

Swap Available: 18572m Swap Used: 938mb Swap Util (%): 30 Reserved: 5532mb

3. this machine is serving as application and reports server. i mean disk intensive applications are running. while buffer cache is 800 mb, rcache is %100. it means
i can still reduce it, right ?

4. as i understand in glance memory window, sys_mem+user mem+ buffer cache+ free mem = physical memory. in which of these is pseudo-swap ?

thanks.

[Steven E. Protter]

Shalom,

Changing dbc_mac_pct does not help with out of memory. Your system is attempting to reserve more swap than it has, hence the error.

Swap should total 1 times memory.

memory is defined in hpux as ram plus swap.

SEP

[John Guster]

in command -mat output:
memory line is pseudo-swap
reserve line indicates swap space(device swap in your case should = RAM size)

pseudo-swap is between 70-80% of RAM(the physical memory) depending on the type of the system

can you post your swapinfo -mat output?

[John Guster]

it cut words...
the command is swapinfo -mat

[Emil Velez · Accepted solution]

When a process starts it uses a certain amount of physical memory for pages of the program as the process accesses pages of memory. As the program requests more memory more memory is given to the program. As the program grows and grows the program gets bigger and bigger.

When a program closes and dies it gives back all of the memory it has allocated for its use.

What I described is virtual memory.. Every program thinks it can grow to 16 TB.


Now we have a limitation.. We cant have all of every program in memory at the same time. So the OS does not have enough physical memory for every program. So when the OS runs out of memory it must page out memory that has not recently been accessed so when there is less than 64 MB of physical memory avaialble vhand starts paging memory to a swap device.

Now what that means is that we have to have enough swap space to put pages of programs that have not been used recently.

So this means that before allowing a program to allocate memory the OS makes a decision whether it has swap space just in case the OS gets low on physical memory. This is swap reservation.

When you start a process as that process uses memory or as a process grows in its memory needs it must decriment (reserve) swap space.

This means that I need swap space enough for all of the programs that run and how much memory they will ever need.

so if I get low on swap space even when I may be swapping a little I could run out of swap.

Solution: provide a system with lots of memory lots of swap maybe twice as much swap as memory.

So this means that a system that uses all of the memory on a system will need at least as much swap space as it has memory.

Pseudo swap is a technology we have on HPUX to make the OS think it has more swap than the system really has... We use this on large systems where we might swap a little but do not need to have lots of disk space that does not get used.

This means a system with 8 GB of physical memory and 12 GB of real swap space with pseudo swap could run about 18 GB of programs before running out of swap.

Now swapping could happen earlier than what I suggest because memory could be used by buffer cache thus starting to page early. If memory is allocated for buffer cache then it cannot be used by processes. Therefore you need to reduce buffer cache to 1 GB or less especially to allow programs to use memory.

I hope this simplified explination makes sense.

good luck

[TommyT]

Thanks Emil, that was the best/simplest explaination I have ever seen about how memory reservation works.
A followup question.. So the dbc_max_pct is actually telling how much memory applikations/processes can use in the physicl memory before it's being paged out?

Thanks again for the explanation.

//Tommy

[Kenan Erdey]

hi again;

no tommy, dbc_max_pct indicates the percentage of physical memory to use for buffer cache.

emil, good explanations but not my questions's answers.
1. kernel parameters that affect sys_mem in glance ?
2. why pseudo swap is seen as used when there is enough place to reserve in device swap ?

[Wouter Jagers]

1. sys_mem denotes the number of pages -not- available for use as pseudo swap (globals.h)

2. 'pseudo' swap is 'fake' swap, as its name indicates. It's merely a game of counters used to trick the OS, as mentioned earlier.

The term 'used' for pseudo swap doesn't mean the same as it does for device swap, as pseudo swap is never actually used as swap space.

'Used' pseudo swap is either 'reserved' pseudo swap, locked memory pages or memory allocated to the kernel itself.

You are right to think reservations will first be done in device swap, but kernel memory allocation and locked memory will reduce the amount of pseudo swap available for reservation, hence it will show up as 'used', meaning 'not reservable'.

Here's a great article (with a nice example resembling your config) which should explain everything:

http://www4.itrc.hp.com/service/cki/search.do?searchString=KBRC00010136++++&docType=EngineerNotes&searchCrit=allwords&search.x=12&admit=-938907319+1168521233839+28353475&category=c0&mode=id&search.y=9

Cheers
Wout

[Wouter Jagers]

As for 1: if we're talking about the same thing (no glance in front of me), I don't think there are any kernel params which would influence that.

Check the attached document (page 83 and up) for more info concerning swap and all related kernel params and global variables.

Cheers

[Kenan Erdey]

Wouter, i can't access the link, can you attach the document ?

[Wouter Jagers]

Sure thing, find it attached.

Cheers !