System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

swapinfo -tam HIGH (dbc_max_pct=50)

 
Akif_1
Super Advisor

swapinfo -tam HIGH (dbc_max_pct=50)

Hi,

OS:HPUX 11.11
Oracle Application 9.0.4
Problem:Memory usage high
RAM 8GB

My memory usage is very high 94% and i need an expert advice upon this issue,

#If i make changes to configurable parameter "dbc_max_pct" which is 50 now does this effect my Oracle Application Server 9.0.4 production environment.

#In case need to revert back old setting what backup to be taken and how to revert it back.

Note: Find an attach file which highlight top/swapinfo/iostat/vmstat info.

Have a good day to all........

Regard's
T(ogether) E(very one) A(chive) M(ore)
27 REPLIES
SoorajCleris
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hi ,

you 80 % swap is already used, it shows that you have pageouts and there must be a memory crunch.

Before reducing the dbc_max_pct, you can check the usage of the parameter.

Do you have glance installed ?

if yes , check what is the current usage for buffercache.

What is the current buffer cache read write hit ratio, is that utilized?


when it was noticed for first time?

Is there any io issue , disk issue,


If you have kmeminfo, it will tell you where the memory is used (but log a case with HP if you have support )

Regards,
Sooraj
"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity" - Dennis Ritchie
SoorajCleris
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

### sorry 18% swap
"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity" - Dennis Ritchie
Raj Briden
Frequent Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

This problem is usually caused by the new dbc_min_pct and
dbc_max_pct kernel parameters. Your dbc_max_pct was probably
set at the default value of 50%, so it is consuming half of
your RAM over time, thus slowing performance and using memory.
SoorajCleris
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

#In case need to revert back old setting what backup to be taken and how to revert it back. ==>


dynamic kernal parameter deosnt require reboot to take effext.

Regards,
Sooraj
"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity" - Dennis Ritchie
Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Iam unable to get ouput from #glance command, My worried area is Oracle Application Server,when i make any changes to dbc_max_pct.

Regard's
T(ogether) E(very one) A(chive) M(ore)

Re: swapinfo -tam HIGH (dbc_max_pct=50)

>If i make changes to configurable parameter "dbc_max_pct" which is 50 now does this effect my Oracle Application Server 9.0.4 production environment.

Hopefully you'll be able to use more of RAM for your Oracle database. 50% is just too high.

>In case need to revert back old setting what backup to be taken and how to revert it back.

There will be no need to change it back. But you just do the same thing and change it to 50.

>Sooraj: dynamic kernel parameter doesn't require reboot to take effect.

This is 11.11, I don't think they are dynamic.
Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hello Dennis,

I appreciate your response to my query, I will go ahead with the dbc_max_pct=20 of 8GB.

Hope while rebooting it will not effect any process.

does a /stand/vmunix backup is a safety to revert back incase of any failure.

Why i am more concern is that , this is a prodution server and it is building new kernel while making changes to dbc_max_pct.

Regard's
T(ogether) E(very one) A(chive) M(ore)
SoorajCleris
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hi Dennis,

I was in a doubt ( so I put a common sentence :) ). I know in 11.23 its immediate. But I couldnt loggin 11.11 and check.

Thanks :)
"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity" - Dennis Ritchie
SoorajCleris
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Yes,

booting from previous kernel will give you old settings.

1. It is recommended to take latest ignite before you make any changes.

2. Its always better to take a reboot before making any changes to ensure the server is booting fine currently. This will help you to isolate if any issues after the change.

Regards,
Sooraj
"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity" - Dennis Ritchie
Rita C Workman
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

1. You do not have enough swap set up to begin with. I recommend disk. Add two 4Gb lvols, and set each up as swapspace. I don't care if they are the same 8Gb disk.
2. If you left dbc_max_% at 50 (default) you likely left the rest defaulting.
20 on dbc_max is fine, you could probably go down to 15 safely.
Set dbc_min to 5
------
Run sar -v 1 20
Look at the output of inode (current/parm)
Set parm ninode down to something more realistic based on the output of 'sar'. For my oracle systems I have them around 4096.
-------
maxdsiz
I don't care what Oracle says...bet you can set that one back too! I have mine at 0X10000000 (268435456). They just need it turned up when they load s/w, but you do not have to keep it there.
-------
vx_ninode.
It's probably at zero (0), which means systems on 11.11 is controlling. It doesn't do it all that well. Try setting at 20,000 to 40,000. I have yet to hit the limit at 40,000. If you have 8Gb memory, this parm creates a huge table - and you're just wasting space.

ninode - creates a round-robin table that recycles inodes. That sar command will show you how much you need. Allow enough room for nightly batch processes if applicable. The default value the kernel will build is way-way bigger than you need. Thus, you are wasting memory.
vx_ninode - Again with too big a table and a version that the system doesn't manage that great. Set the value, so you don't waste mem.
dbc_max & min_%. These two default parms have been a known issue for years. There are a multitude of threads on this. Do a SEARCH, and you will be amazed.

These are just a couple of default parms to look at and fix right away. There are others, but it is best to just adjust as few as possible.

Remember -
Tuning is a journey taken in small steps.

Regards,
Rita
Rita C Workman
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

On that immediate change and oracle.

11.11 has very few parms, dbc_max_% will require a kernel rebuild, hence a reboot.

11.23 for dbc_max_% (and min) can be changed immediate - or next reboot.

11.31 for dbc_max_% is automatic and can be changed immediate.

As for Oracle...as when any reboot is required, just shut it down, do your parms (selectively), then bring it all back up. By doing it one parm at a time you can ensure that each parm works for you...or change it back. If Oracle comes down clean and is set up properly, you should be okay.
In fact - with the parms tuned better, Oracle should run better!

Rgrds,
Rita
Steven E. Protter
Exalted Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Shalom,

dbc_max_pct" which is 50 is the buffer cache.

Setting this to 50% is very high and allocates half of available memory to this cache.

You can substantially lower system memory use by setting this number to a lower figure. Databases like Oracle do a better job than the Os caching data, especially on the 11.11 OS.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hi Rita,

Find below sar output:

xxxapp1:/ ABC#sar -v 1 20

HP-UX xxxapp1 B.11.11 U 9000/800 04/27/10

08:42:19 text-sz ov proc-sz ov inod-sz ov file-sz ov
08:42:20 N/A N/A 954/4096 0 2223/34816 0 15444/63498 0
08:42:21 N/A N/A 954/4096 0 2224/34816 0 15433/63498 0
08:42:22 N/A N/A 954/4096 0 2223/34816 0 15432/63498 0
08:42:23 N/A N/A 952/4096 0 2224/34816 0 15425/63498 0
08:42:24 N/A N/A 952/4096 0 2225/34816 0 15434/63498 0
08:42:25 N/A N/A 953/4096 0 2224/34816 0 15442/63498 0
08:42:26 N/A N/A 954/4096 0 2225/34816 0 15451/63498 0
08:42:27 N/A N/A 952/4096 0 2226/34816 0 15462/63498 0
08:42:28 N/A N/A 951/4096 0 2223/34816 0 15465/63498 0
08:42:29 N/A N/A 954/4096 0 2226/34816 0 15469/63498 0
08:42:30 N/A N/A 954/4096 0 2225/34816 0 15453/63498 0
08:42:31 N/A N/A 953/4096 0 2227/34816 0 15447/63498 0
08:42:32 N/A N/A 952/4096 0 2224/34816 0 15446/63498 0
08:42:33 N/A N/A 953/4096 0 2224/34816 0 15436/63498 0
08:42:34 N/A N/A 950/4096 0 2224/34816 0 15425/63498 0
08:42:35 N/A N/A 950/4096 0 2225/34816 0 15411/63498 0
08:42:36 N/A N/A 949/4096 0 2225/34816 0 15409/63498 0
08:42:37 N/A N/A 950/4096 0 2223/34816 0 15394/63498 0
08:42:38 N/A N/A 947/4096 0 2223/34816 0 15386/63498 0
08:42:39 N/A N/A 950/4096 0 2224/34816 0 15401/63498 0
T(ogether) E(very one) A(chive) M(ore)
Raj Briden
Frequent Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

You can go ahead and reduce the dbc_max_pct parameter to the value 25 and check the performance.

Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hello Raj,

My concern mainly with this two areas.

#Does this parameter effect oracle application server.
#How to revert back if kernel fail to startup server after new kernel buildup.

Regard's
T(ogether) E(very one) A(chive) M(ore)
Raj Briden
Frequent Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

i had reduced this paramaeter with out any issue.

Reboot the server before reducing this server to check server is booting fine.

Always have a good ignite backup!

Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hi All,

Even though after dbc_max_pct=20 value changed still memory usage is high.

The http process and forms connection increase considerably.

Need help to solve the issue.


Rgd's
T(ogether) E(very one) A(chive) M(ore)

Re: swapinfo -tam HIGH (dbc_max_pct=50)

>Even though after dbc_max_pct=20 value changed still memory usage is high.

You can decrease it to 5%.
Are you doing lots of swap outs?
Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hello Dennis,

You can decrease it to 5%.
Are you doing lots of swap outs?

#yes But my 8GB memory only have 800MB + /logs/paging 800MB swap which not possible to increase due to lack of FS space.

Any suggestion
T(ogether) E(very one) A(chive) M(ore)

Re: swapinfo -tam HIGH (dbc_max_pct=50)

>yes

Yes to what? Have you changed dbc_max_pct to 5%?

>But my 8GB memory only have 800MB + FS 800MB swap which not possible to increase due to lack of FS space.

If you can't afford more device swap, you'll have to stop running so many processes.

I assume this is related to your other thread:
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1425598

Why do you have so many f90webm processes?
Akif_1
Super Advisor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Hi Dennis,

I appreciate your valuable response.

Almost 500 users connect Oracle application - vis - Oracle database.

The "/u01/midhome/bin/f90webm server webfile=HTTP-0,0,0,cfg2" create each process whenever user connected.

ex:500 users create 500 process of above stack.

Regd's
T(ogether) E(very one) A(chive) M(ore)
Rita C Workman
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Reduce your ninode to 4096. That is more than enough...
Also recommend setting vx_ninode to anything from 20,000 to 40,000.

What's your maxdsiz & maxssiz set to?
Remember, Oracle wants maxdsiz somewhat large, but I'm running 10.2, (8Gb mem) and performance runs fine with maxdsiz turned back down. We have about 500 folks hitting it here, plus external querries constantly being run by folks via web. No performance issues.

One last thing...if it's showing a high memory usage, that isn't necessarily a bad thing. It just means that memory is busy. You know you have an issue if you 'page out' alot. So instead of swapinfo, what is vmstat showing. Clear your counters using:
vmstat -z

Then monitor it by using the vmstat -nS 1 10 periodically. If you are seeing alot of page-outs then get back with us.

Rita
Bill Hassell
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

>> Even though after dbc_max_pct=20 value changed still memory usage is high. The http process and forms connection increase considerably. Need help to solve the issue.

Adjusting dbc_max_pct and changing the ninode parameter might reduce memory usage by a few megs. There are only three things that will help:

1. Reduce the number of users.

2. Tell your Oracle DBA to reduce the size of SGA by half.

3. Double or triple the amount of RAM in your system.

Simply put, your system is working at its maximum which is a good thing unless performance is not acceptable. Choice #1 is probably not practical. Choice #2 will work but performance will likely do down quite a bit. #3 is the solution. Your system has no room for growth and is simply undersized.


Bill Hassell, sysadmin
Rita C Workman
Honored Contributor

Re: swapinfo -tam HIGH (dbc_max_pct=50)

Did you put up any additional disk swap?
---------------------
There are additional things to tune that often folks don't want to hear about.
Tune your Oracle querries!

That is something developers & sometimes DBA's often don't want to be bothered with, but find which querries are causing the highest impacts. Have your DBA's to run the tools to find these and then (if they are worth their salt..) have them recommend some improvements on the statements. One bad querry being run by alot of folks can cause alot of issues. By cleaning up just a few bad ones can be beneficial for everyone.
Another thing you may find is that a querry isn't cleaning up (releasing resources) properly, hence slowing performance.

Another thing to look at is where are your Oracle logs being written to? which disk?
If you are allowing redo logs and cntl logs and archive logs, all hitting the same disk, you will have performance issues too. Granted this should show up on disk I/O, but waiting on the overhead I/O will cause other things to slow down.

My point is......look at performance issues as a 'whole'.