Mmemory settings

[John Jayaseelan]

hi,

The current settings are
dbc_max_pct 50
dbc_min_pct 5


What is the impact of setting these values to on HP-UX 11.0 system.
dbc_max_pct 5
dbc_min_pct 3


Thanks
John Jayaseelan

[Shaikh Imran · Accepted solution]

Hi,
dbc_max_pct = dynamic buffer cache settings.

Dynamic buffer cache works well in environments below 4GB of RAM,but can waste memory and even negatively impact NFS performance if allowed to grow too large. The default setting in 11i is 50% for dbc_max_pct,it's best to limit this to 20% or at most 800MB of RAM.
In 11.0 it's even more important; limit it to 10% or 400MB, whichever is the lower. Despite the positive effects of a file system buffer cache,Oracle ignores it when writing, because it opens its datafile with the O_DSYNC flag,all writes are only acknowledged when written to disk, or, in our case, the filers NVRAM.
Oracles' own SGA (system global area) handles shared pools of memory directly and
allocates db block buffers in the SGA controlled in the init.ora file.

It's generally recommended to try both static and dynamic cache.
However, depending on your type of database application, you may find that static
buffer cache (setting "nbuf" and "bufpages" to non-zero values) reduces "vhand"
process activity from stealing pages and thereby increasing load on the CPU in
order to dynamically adjust FSBC.

Make sure you know the size of the SGA and the static buffer cache before running any tests or production
system on your host. A 4:1 ratio is a good starting point in observation of the aforementioned limits.
If your application demands significant file-system I/O you may consider decreasing the ratio to 3:1.Oracle will allow you to configure a larger SGA than physical memory.
Heavy swapping activity could be the result, but this may not be noticed until the database starts thrashing.

Hope this will help you analyze your requirement.

Regards,

[Fred Ruffet]

Don't know if you're using Oracle, but as long as Shaikh talks about it...

Oracle uses its own cache and may imply a huge overhead with HP-UX cache. What Oracle recommends is to bypass the OS cache for FS where DB files are. Those FS must be mount with those options :
log,nodatainlog,largefiles,mincache=direct,convosync=direct

Regards,

Fred

[John Jayaseelan]

Hi,

Thanks for the reply. The databases on the system are Informix, Progress & Universe.

Following is the last lines of glanceplus with 125 users on the HP-UX 11.0 system.

Total VM : 710.0mb Sys Mem : 152.5mb User Mem: 1.75gb Phys Mem: 3.75gb
Active VM: 166.1mb Buf Cache: 1.31gb Free Mem: 552.8mb


Please provide your suggestions.

Thanks,
John Jayaseelan

[Fred Ruffet]

Settings you mentionned (dbc_max_pct=5 and dbc_min_pct=3) may be a good thing, but I don't know about those DBMS. All I can say is that having 500Mo of free mem can be cool.

Does it happens that your machine uses swap ?
It could be usefull to monitor memory/swap and disk usage. This will be the only good indicator on wether to set those parameters or not.

Regards,

Fred

[John Jayaseelan]

Fred,

Is there any other way to montor the memory/swap disk usage. The following glance output is uncertain, it shows the 'swapp util' avg as 42% but it has not used any swap space at all.

B3692A GlancePlus B.11.01 14:49:23 ccprod01 9000/898 Current Avg High
--------------------------------------------------------------------------------
CPU Util S SNNARRU U | 68% 40% 100%
Disk Util F F | 25% 26% 100%
Mem Util SSU UB B | 85% 85% 86%
Swap Util U UR R | 43% 42% 43%
--------------------------------------------------------------------------------
SWAP SPACE Users= 119

Swap Device Type Avail(mb) Used(mb) Priority
--------------------------------------------------------------------------------
/dev/vg00/lvol2 device 1280 0 1
/dev/vg00/lv_swap2 device 1280 0 1
/dev/vg00/lv_swap3 device 2200 0 1
pseudo-swap memory 2992 1810 na


Swap Available: 7752mb Swap Used: 1810mb Swap Util (%): 43 Reserved: 3329mb


Thanks

[Shaikh Imran]

Hi,
Maybe you have used it earlier but,

I think you should use
swapinfo -m
for swap space usage in MB
vmstat
For virtual memory statistics.

see man of the above commands.

Regards,

[John Jayaseelan]

Hi,

The glance output is uncertain, it shows the 'swap util' avg as 42% but it has not
used any swap space at all.

Could you please tell why?

John Jayaseelan

[Geoff Wild]

Swap utilization - it's just reserving space for processes...

dbc_max_pct - you want that to work out to something between 350MB to 650 MB of ram.

For example, if you had 2GB of ram installed, then set it between 18 and 32.

Rgds...Geoff

[John Jayaseelan]

Geoff,

Thanks for the update.

The available memory is 768MB.

I plan to set the dbc_max_pct = 10 & dbc_min_pct = 3. Is it fine?

Thanks

[John Jayaseelan]

Geoff,

I have recevied the following results using the memdetail pgm you posted in the past. Also I have listed results from swapinfo. The device swap differs from the results.

Please explain what are these values active virtual ,active real & memory swap.

# memdetail
Memory Stat total used avail %used
physical 768.0 752.6 15.4 98%
active virtual 200.9 88.5 112.4 44%
active real 88.8 12.2 76.6 14%
memory swap 534.3 534.3 0.0 100%
device swap 768.0 622.3 145.7 81%


# swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 786432 197184 589248 25% 0 - 1 /dev/vg00/lvol2
reserve - 439104 -439104
memory 547112 547112 0 100%


Thanks,

[Bill Hassell]

> Following is the last lines of glanceplus with 125 users on the HP-UX 11.0 system.

> Total VM : 710.0mb Sys Mem : 152.5mb User Mem: 1.75gb Phys Mem: 3.75gb
> Active VM: 166.1mb Buf Cache: 1.31gb Free Mem: 552.8mb

So you have 3.75Gb of RAM but 1.31Gb if buffer cache--way too much, especially at 11.0 since the 'dynamic' part of the buffer cache is very stubborn and doesn't shrink very well. Change dbc_max_pct to 10 or 15. When the buffer cache is very large, it actually slows down file open/close and cache accesses due to the somewhat serial nature of buffer cache access. 11.11 changed this completely so access is a hashed list and dynamic changes can occur very quickly (within seconds).

Since you are running Universe, that database needs massive numbers of files opened at the same time for good performance. In the Universe config file, the MFILES value indicates how many files can be opened at the same time. Universe benefits greatly by increasing this to several hundred (250, even 500) and you will see a dramatic decrease in HP-UX system overhead. The reason is that you will reduce the number of open and close system calls made by Universe. Make sure you calculate a new nfile kernel value. Just for Universe, you'll need Uinstances * MFILES. So if you run 100 Universe sessions with MFILES=500, you'll need 5000 plus the rest of the system usage for nfile. A rough guess for the rest of the system would be nproc * 5, but use sar -v 1 to be sure.

[Geoff Wild]

I'm a little confused - are you showing results from 2 different systems?

Glance showing 3.75 GB of ram and memdetail 768MB...

memdetail shows in MB, swapinfo in KB...

Run the attached script - called hpmem.

Bill has given you a good explanation on what is happening.

Rgds...Geoff

[John Jayaseelan]

Geoff,

The 3.75GB is production system, 768MB is test system.


The result of hpmem from the test system

# /home/scripts/hpmem.sh
HP-UX cctest01 B.11.00 A 9000/810 2004826925 two-user license
CPU Count: 1
CPU Speed: 180 MHz
CPU HW Support: 32/64-bit
Kernel Support: 32-bit
RAM Size: 768 MB
bufpages: 323 MB
maxuprc: 1024
maxvgs: 10
maxfiles: 60
max_thread_proc: 64
nfile: 4146
nflock: 200
nproc: 2068
ninode: 2492
shmmax: 1073741824
shmmni: 200
dbc_max_pct: 50

Thanks

[Geoff Wild]

Okay - perfect.

As Bill says, for production, 10 to 15 for MAX_DBC_PCT

On Test, you can't save much from default - try 35 to 45.

Rgds...Geoff

[John Jayaseelan]

Geoff,

The following are the results in MB from memdetail & swapinfo.

The device swap usage differs. Please tell what are the 'active virtual' & 'active real' components.

# memdetail
Memory Stat total used avail %used
physical 768.0 740.3 27.7 96%
active virtual 174.8 161.8 13.0 93%
active real 159.8 151.8 8.0 95%
memory swap 533.6 348.5 185.1 65%
device swap 768.0 499.1 268.9 65%
[cctest02]: /root
#
[cctest02]: /root
# swapinfo -m
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 768 102 666 13% 0 - 1 /dev/vg00/lvol2
reserve - 397 -397
memory 534 349 185 65%

Thanks

[Geoff Wild]

From glance:

Total VM : 342.3mb Sys Mem : 232.0mb User Mem: 206.9mb Phys Mem: 768.0mb
Active VM: 221.8mb Buf Cache: 192.0mb Free Mem: 137.1mb

# memdetail
Memory Stat total used avail %used
physical 768.0 630.9 137.1 82%
active virtual 342.3 221.8 120.5 65%
active real 185.2 130.5 54.8 70%
memory swap 555.3 349.8 205.5 63%
device swap 1024.0 298.4 725.6 29%


Active Virtual = Total Virtual from glance

Used Active Real = Active VM from glance


So, Active Virtual is:
The total private virtual memory (in
KBs unless otherwise specified) at
the end of the interval. This is
the sum of the virtual allocation of
private data and stack regions for
all processes.

And , Used Active Real:
The total virtual memory (in KBs
unless otherwise specified)
allocated for processes that are
currently on the run queue or
processes that have executed
recently. This is the sum of the
virtual memory sizes of the data and
stack regions for these processes.

Rgds...Geoff

[John Jayaseelan]

Geoff,

Thanks for the reply.

How the value for the 'active virtual' is determined?

Please tell what is 'Total - Active real' & 'Used - Active real' and how this value is determined.

From the results provided from memdetail & swapinfo -m, the device swap usage differs. Why.

From the following glance output, pseudo-swap is used when there is free space available on device swap. But pseudo-swap is supposed to be used when there is no free space available on device swap. Please confirm.


Swap Device Type Avail(mb) Used(mb) Priority
--------------------------------------------------------------------------------
/dev/vg00/lvol2 device 768 43 1
pseudo-swap memory 548 311 na





Thanks

[Geoff Wild]

Device swap is same, on mine:

# memdetail
Memory Stat total used avail %used
physical 768.0 629.3 138.7 82%
active virtual 337.2 177.7 159.5 53%
active real 183.4 104.7 78.7 57%
memory swap 555.3 349.8 205.5 63%
device swap 1024.0 293.3 730.7 29%

# swapinfo -m
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 512 27 485 5% 0 - 1 /dev/vg00/lvol2
dev 512 27 485 5% 0 - 1 /dev/vg00/lvol3
reserve - 239 -239
memory 555 350 205 63%


device swap used from memdetail: 293.3 which is the same as the total of the 2 dev's and reserve from used in swapinfo -m.

Yours is the same as well:

device swap used from memdetail: 499.1
reserve + dev from swapinfo -m : 499


As far a pseudo swap - that is used before device swap.

Rgds...Geoff

[John Jayaseelan]

Geoff,

Thanks for the reply. I thought the 'device swap' is from 'dev' only not from 'reserve'.


How is the value for the 'active virtual' determined?

what are 'Total - Active real' & 'Used - Active real' and how this value is determined.

Thanks

[Geoff Wild]

John,

You're starting to go too deep for me :)

I'm looking in the source code now,

/* get dynamic information about the system */
if (pstat_getdynamic(&dbuf, sizeof(dbuf), (size_t)1, 0) != -1)
{
avail_mem = dbuf.psd_free;
total_virtual_mem = dbuf.psd_vm;
active_virtual_mem = dbuf.psd_avm;
total_real_mem = dbuf.psd_rm;
active_real_mem = dbuf.psd_arm;
}

That's how it's determined.

From swapinfo man:

dev Paging space residing on a mass storage device,
either taking up the entire device or, if the
device contains a file system, taking up the
space between the end of the file system and
the end of the device. This space is
exclusively reserved for paging, and even if it
is not being used for paging, it cannot be used
for any other purpose. Device paging areas
typically provide the fastest paging.


reserve Paging space on reserve. This is the amount of
paging space that could be needed by processes
that are currently running, but that has not
yet been allocated from one of the above paging
areas.

So, memdetail adds those together.

Rgds...Geoff