Experts...Can Use Your Tips On Tuning Large Memory Server

[Rita C Workman]

Thanks Jim! Got it.

Now I have a bit of info on NUMA, and I know how to turn it on. Unfortunately, no where in anything do I recall it give any insights as to what to set some static parms, like filecache_max/min; maxdsiz; maxssiz; shmmax for few. In fact, it doesn't give any parm recommendations that I can recall from my reading the last few days. But it does say that you either go with dynamic configuration or NUMA optimization, but not both.

So...my question would now say:

If they don't go with grid technology and NUMA and they opt to stay with ILM (Interleaved memory).....are there any tips for parm setting you folks might recommend I watch out for with this box?

Thanks,
Rita

[Steven E. Protter]

Shalom Rita,

Back online after a long break.

filecache_max/min;
set these two as you would on older HP-UX. Note that Oracle does sometimes benefit from file cache so you might find increase does help.

Oracle actually dictates some of this in the installer, and violating those rules is done at the peril of the sysadmin. shmmax needs to be equal to physical memory even though it can not possibly be that large.
maxdsiz; maxssiz; shmmax

Regards,

SEP

[Zinky]

Since it appears you will not be partitioning your rx8640 - there's less of an issue sticking to the "rules" on CLM/ILM, processor/cell localisation,etc -- as set forth in the HP and Other Whitepapers.

I *would* focus "tuning" and turbo-charging your Oracle DB Server environment on two things:

1) Processor binding to certain Oracle Processes
2) FastIO! (aka RAM Disk) for key

I would also advice your DBAs to go with ASM Storage -- preferably 2 or 3 ASM Diskgroups instead of using traditional RAW or CookedIO -- no matter what the overhead projector says on improvements with VxFS/DirectIO.

[Rita C Workman]

Thank you all for your responses. Going to hold the thread open for a day or two longer and then will give points.

Rita

[Zinky]

Rita Ma'm,

Pls. also be aware of:

http://skrajend.blogspot.com/2008/09/numa-after-10204-upgrade-is.html

In case you end up your DB is not using all available CPUs.... That is IF you're on 10Gr2

[Zygmunt Krawczyk]

Alzhy wrote:
"I would also advice your DBAs to go with ASM Storage -- preferably 2 or 3 ASM Diskgroups instead of using traditional RAW or CookedIO -- no matter what the overhead projector says on improvements with VxFS/DirectIO."

From performance point of view the ASM is good, but not the only solution. Comparable performance can be achieved on filesystem with Oracle Disk Manager (ODM).

Read the doc:
"Improving the performance of single instance Oracle on file systems: Analyzing the impact of Oracle Disk Manager (ODM) â a feature of HP Serviceguard Storage Management Suite"
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01917294/c01917294.pdf

[Zinky]

Zyg...
I won't disagree with ya but it is really what the DBA and SysAds are comfy about and which offers the best manageability and less risks. WIth ASM, there is no filesystem/mount point to corrupt, no "volume manager layer" to deal with. Sure with ASM a DBA has to maintain a small DB Instance -- +ASM that acts as manager of provisioned disks - but compared to traditional storage solutions - this is a whole lot simpler and moves the responsibility of manaing and monitoring DB storage to our DBA Amigos...


Say -- is ODM an evolution from Veritas Database Edition sir? Does it use Veritas bits (i.e,. FIlesystem?)

We've used Veritas DBED in the past but moved most of our DBs to ASM on HPUX (and ASMLib flavoured ASM on Linux) -- practically zilch issues compared to when we were on sashimi (raw) and sushi (cooked filesystems with DirectIO).

Cheers!

[Rita C Workman]

Alzhy,

Thanks for the blog pointer. It mentions his problem with only seeing 3 CPU, but it doesn't go into detail though.
For example:
On HPUX NUMA is turned off by default, and he does not mention if HPUX was properly configured for NUMA.

On the Oracle side NUMA, depending on version is controlled through an initialization parmater and only checked at startup time! On 10gR2 and 11gR1 I think it's called "_enable_NUMA_optimization". I don't think even Oracle likes this parm. Now on the 11gR2 there is a new one "_enable_NUMA_support" that Oracle supports.
So on startup, even though Oracle may have the right startup/enable NUMA parm set as "on" for the 10g and later versions - with a caveat that folks miss. Oracle NUMA will first go out and check if the server is set up for NUMA. If it is, then it proceeds with it's next steps to confirm allowing optimization. If the HPUX is NOT set (i.e. LORA_MODE=0), then Oracle NUMA is disregarded and it's optimizations will be disabled.

There's a bunch of other parms and settings that Oracle must give attention to, that clearly in his blog they obviously didn't...but what all of this has shown is that doing a ccNUMA or NUMA configuration requires DBA's & SysAdmin's to work together.
Introducing a whole new concept b/t DBA's and SysAdmins, cause Grid Technology is not for the timid.

Regards,
Rita

[Rita C Workman]

Almost forgot...on the storage thing (ASM Storage), well each to his own on this.

For myself, I will try to be open to the notion of trying/testing Oracle's disk management utility....but at this point in time I can not say that I am in anyway moved to use it in any production environment.

FYI...I was actually a DBA for a very short time, and the one thing it taught me was that I knew nothing about Infrastructure. I say leave HPUX and Storage to the professionals...and unless the DBA is a certified HPUX and certfied Storage person-they shouldn't play with what they do not know.

Just my thoughts,
Rita

[Rita C Workman]

Very much appreciated the pointers to the info on NUMA.

Was hoping to get more parm advice in the event they go with ILM and not NUMA. Although according to one document, Oracle actually has no way of knowing if the box is set up to run with NUMA or not. Now, isn't that interesting!

Guess for tuning parms, if we go ILM (native) I'll have to go by the standards I'm used to and 'tweek' from there.

Kindest regards to all and Many Thanks!
Rita

This thread is now closed.