Help, need memory or more swap?

[Lisa Mauer]

Hi all:
I have an rp8400 that is hard partitioned running close to 8 Oracle database plus numerous application software on each. My manager doesn't believe me when I say we need more memory.... He thinks VM and swap can go on forever and ever and we never need more memory. We need to add 4 more Oracle databases to this server and to the other partition, which looks worde then this one...I am attaching some information from my server, can someone either please help me explain in easy terms that we need more memory or hey - tell me I'm wrong and that'll be fine too :) I do not see any page-outs in glance. I have gone through to the best of my ability and tuned this system, but I'm also a software engineer, so most of my time is spent supporting programmers, not being a Sys Admin... So anything you see, please let me know.... Thanks!

[Pete Randall]

Lisa,

You don't seem to be paging/swapping at this point, but another 4 Oracle databases could well push you over the edge and into swapping. Any hopes of decent performance at that point will be just that: hopes. I can't really think of a way to anticipate just how these additional DBs are going to impact your memory, you may have to just implement them and observe.


Pete

[John Payne_2 · Accepted solution]

Each Oracle DB has a SGA size. This SGA is real memory. Once you allocate your SGA's overthe limit of free memory, you are out of memory and end up with real swapping, assuming the SGA's are sized properly for the database. (Like Pete said.) I tell my DBA's that their available memory for SGAs is the total system memory minus what is used with oracle down. When they go over, they have problems, and there is no one to blame but themselves. (IT's amazing what a little training can do sometimes...)

If you make a chart for you manager with total memory at the top, then go down the list for used (without oracle) then existing DB's with their SGA sizes. MAke a column where you can do a running subtraction until you get to the 80MB available level you are at. Add a space, then list the 4 new DBs with their expected SGA size. Subtract this in the total memory available column to get a negative number. In my experience, this is usually pretty effective with manager-type people. (As long as there are no shiny sections of your chart for them to get distracted by...)

Overallocating your SGA's on a machine is a bad thing. It makes performance unpredictable, at the very least.

Hope it helps
John

[RolandH]

First of all I would recommend to set dbc_max_pct/dbc_min_pct to a lover level, because the oracle db has it´s own buffer cache.
See this thread for that problem.
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=750342

Add the space needed for each SGA of your oracle db´s. Does it exceed your physical memory I would recommend to buy more.

Ro

[TwoProc]

OK, everyone who doesn't have a boss that does this to them - go over and thank them for being your boss!! :-)

[Bill Hassell]

The boss is correct: add lots more swap space, perhaps another 16 Gb (you'll need to change maxswapchunks for the larger swap space). Now tell your boss that you can run another 10 or 20 instances of Oracle. HP-UX is indeed a virtual memory system so you can run 16Gb of processes in just 2Gb of RAM.

Oh, I hope there are no concerns about performance. WIth a few dozen users, transactions will take a dozen minutes to complete. But think of all the $$$ you saved by not providing lots of RAM for the Oracle processes.

Seriously, you add RAM when the page-out rate moves into double digits or more (vmstat). When po reaches triple digits, users and DBAs will complain. Only RAM (or a reduction in Oracle instances) will fix the performance issues.

[Steven E. Protter]

Your boss needs to read Bill's post carefully.

Memory is much, much, much, much, much faster than disk.

The minute you consistently see actual paging, you will need more memory. Swapping will make everyone nuts.

Your firm paid a lot of money for an rp8400 server. I wish I got to work on such hot hardware.

So now we're going to scrimp on memory. Ridiculous.

Tune the oracle sga's. Get rid of unneeded programs and daemons. Run Bastille and it will eliminate unsafe daemons and improve peformance.

Then get the memory.

If you're really not a sysadmin, you may want to bring in someone to do some analysis and tuning on the system. It may be worth the price.

Or you can bring in my scripts, which are attached. You can collect data over a long timer period, batch, online whatever and take a look at the trends and summary data. Its a very helpful script set, originally given to me without warranty by HP. I've done some upgrades and fixes on them.

SEP

[Rita C Workman]

Hello Lisa,

Yes Bill and Stephen have spoken. But do not ignore what Roland mentioned. Your dbc_max_% is way too high. Of 8Gb RAM your taking nearly 1.9Gb for system. That is far more than you need. Reduce the dbc_max to half of what you have it now and see how that performs. Makes me wonder if there are a couple others you could look at:

Like doing a sar -v 1 10 and see if your inode is set far more than it needs to. I keep on small systems to 2048 and the largest I have is only 8192.

The other one is it's cousin vx_ninode, bet you have it set to 0. Consider changing this cause 0 causes some ridiculous cache table to be set, like 250,000. Might start with anywhere from 20,000 to 40,000 and monitor from there.

And obviously I have to agree with the 'masters'. Here is one point you might share with your boss. Your running 8 Oracle instances now....makes me wonder what version of Oracle. Cause when they started into 8i-I noticed it grabs resources. Actually that's the argument I used to get rid of the K boxes and upgrade to (don't read this Stephen...) the new 8420's that we will plug up this week.

Rgrds,
Rita

[harry d brown jr]

I agree with Roland and Rita, LOWER dbc_max_pct to 5 and dbc_min_pct to 2. With oracle db's you don't need a lot of filesystem buffer cache.

live free or die
harry d brown jr

[Lisa Mauer]

Thank you very much for you answers! I am debating sending him the link to the posts :) I will definatley chnage the db min and max, I saw that when I was putting my message together :) I appreciate it very much! Lisa

[Bill Hassell]

I'm sure your boss means well and has studied the Unix for Managers book very carefully. But you might want to summarize the responses or point him to the HP-UX Performance book (HP-UX Tuning and Performance: Concepts, Tools and Methods by Robert F. Sauers) where the details are found. On any server, swapping means very bad performance. PCs can get away with swapping because they are typically used for only one thing at a time.

For Oracle, performance can be maximized by extending SGA (the shared memory area) large enough to allow complete row insertions and to avoid disk-based tempfile sorting (and other tasks). A good DBA will use the Oracle tuning tools to see if new indexes are needed or if indexes have become unbalanced by looking at SQL statements that take a long time to run. Large SGA's for each instance (2-4Gb) can really improve performance but will be crippled by swapping. And of course, if Oracle is only a 32bit version, you'll have lots of challenges getting SGA larger than 960 megs.

Both the machine and the instances of Oracle cost a lot of dollars. Using swap instead of more memory is like putting skateboard wheels on a Corvette and expecting high performance.

[Borislav Perkov]

Hi Lisa,
You can also make adjustments on your system.
Look the /etc/services.window file.
Regards,
Borislav

[Ted Buis]

OnLineJFS could also help, by avoiding double buffering in Oracle and buffer cache. See the attached cookbook. Remember that access to RAM is roughtly 1000 times faster than disk, and queues only go so deep. Also, you must be careful that you don't get totally I/O bound. You need to consider if you have additional I/O capacity in addition to memory and CPU as you add jobs.