Experts...Can Use Your Tips On Tuning Large Memory Server

[Rita C Workman]

Tuning is something I enjoy doing. I've worked on these kinds of boxes plenty of times, but generally with less memory.

This particular box is an rx8640 running 11.31 Itanium. It has 95.95Gb of memory, plus swap:
lvol2 = 10Gb, plus 6 more 8Gb disks for additional swap. Lots of CPU.

Running Oracle apps.

So, are there any caveats or tips anyone could share for tuning so much power. I don't want to increase a parm, just because I have so much, and create a waste of resources.
I want to be sure that this much memory, doesn't call for different methodology in tuning.

Many thanks,
Rita

[Zygmunt Krawczyk · Accepted solution]

You can consider tune the memory for Cell Local Memory (CLM). Read the docs:

The Oracle Database on HP Integrity servers
http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA2-0547ENW.pdf

Running Oracle Database 10g or 11g on an HP-UX ccNUMA-based server
http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA2-4194ENW.pdf

[Zinky]

Is it running as a single nPar?
By "Oracle Apps" - what do you mean?
- Database?
- Fusion?
Have you any idea of what the overall memory requirements are for your "Apps"?

Do you see any "surplus" on memory?

[Torsten.]

I think you need to tell some more details to get some good hints ...




Regarding "...so much power", what does "machinfo" show?


Examples:

# machinfo
CPU info:
8 s (1.6 GHz, 20 MB)
4794 MT/s bus, CPU version 4
64 logical processors (4 per socket)

Memory: 981492 MB (958.5 GB)

Firmware info:
Firmware revision: 002.074.000
FP SWA driver revision: 1.18
IPMI is supported on this system.
BMC firmware revision: 1.02

Platform info:
Model: "ia64 hp Superdome2 8s"
...

and


CPU info:
64 Intel(R) Itanium 2 9000 series processors (1.6 GHz, 18 MB)
533 MT/s bus, CPU version C2
128 logical processors (2 per socket)

Memory: 981492 MB (958.5 GB)

Firmware info:
Firmware revision: 7.44
FP SWA driver revision: 1.18
IPMI is supported on this system.
BMC firmware revision: 24.04

Platform info:
Model: "ia64 hp superdome server SD64B"
...

[Innate Ideas]

Kernel parameter base_pagesize = 16 will reduce the CPU overhead for managing all that memory.

We've had good experiences setting base_pagesize = 64 on our HPVM hosts (not HPVM guests) with 96GB of memory. base_pagesize = 16 seems to be optimal for our Oracle application and database environments.

As always your milage may vary.

[Zinky]

Rita,

FWIW - if it is going to be a single nPar, single Instance, Lots of Surplus mem kind of environment -- and IF the env needs superfast I/O for some routine or processing -- I'd allocate a part of Memory as "RAMDISK" -- to me that is the BEST use of memory that would otherwise just be "wasted" in cache.

Exmples of RAM DISK use would be sort areas and TEMP TABLESPACES...

[Rita C Workman]

Sorry folks, been out of town for a couple days, but was able to do some reading on this while on the plane(s)..

Here is the machinfo on the box requested, and the box is I believe running or going to be running Oracle 11gR2.

CPU info:
7 Intel(R) Itanium 2 9000 series processors (1.6 GHz, 12 MB)
533 MT/s bus, CPU version C2
12 logical processors

Memory: 98254 MB (95.95 GB)

Firmware info:
Firmware revision: 9.022
FP SWA driver revision: 1.18
IPMI is supported on this system.
BMC firmware revision: 3.01

Platform info:
Model: "ia64 hp server rx8640"
Machine ID number: n/a
Machine serial number: n/a

OS info:
Nodename: no-name-server
Release: HP-UX B.11.31
Version: U (unlimited-user license)
Machine: ia64
ID Number: 0769464687
vmunix _release_version:
@(#) $Revision: vmunix: B.11.31_LR FLAVOR=perf

Now...what I don't know yet is if the DBA's plan to, or even know about, turn on for Oracle Optimizing using ccNUMA or NUMA.
I'm waiting to hear about that.

Zygmunt's docs were very helpful. But now I'm looking for a white paper called "Dynamic server resource allocation with Oracle Database 10g or 11g on an HP-UX ccNUMA-based server" - and so far I can't find it under the new docs webpage. So if anyone knows where to find this...let me know.

Points will be generously given later,

Many thanks!
Rita

[Zinky]

Oh so a Database. Purely DB hosting?

AFAIK, 10GR2 and 11G are both NUMA aware and have lots of tools to auto-this or auto-that BUT nothing beats a superfast I/O for parts of Oracle that NEED superfast I/O. Your DBA should be able to tell you which -- but TEMP Tablespace and other sort areas or other transient TableSpaces that require super high speed mosdt certinly would be very good use of that excess memory -- that is IF you forecast your 96GB of RAM will have some surplus.

[Rita C Workman]

Thanks Alzhy,

What's I'm seeing is that the server itself is not set up for NUMA (i.e. policy & LORA_MODE show default).
I'm "hoping" to get some info from DBA's or mgmt to clarify what the DBA's plans are, because if they 'think' that all they have to do is flip the switch on their side and it's a done deal, then everyone is in for a surprise.
I know if they turn on Oracle Optimization that I will need to change the settings to ensure it's mostly CLM and the rest ILM. BUT, what else do I need to do to ensure performance- what other parms might I need to tweek.
Cause once they turn it on, you can not easily change parms with CLM - not even dynamic ones with things up and running.

ILM tuning is one thing, but CLM is something I'm such a newbie on.

And I'm having trouble trying to find the white papers I want to review....wish HP would have left well enough alone. Now it's harder than ever to find things.

Thanks,
Rita

[James R. Ferguson]

Hi RIta:

I believe that this is the whitepaper you want:

http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA3-1350ENW.pdf

Regards!

...JRF..