- Community Home
- >
- Servers and Operating Systems
- >
- Operating System - HP-UX
- >
- System Administration
- >
- Experts...Can Use Your Tips On Tuning Large Memory...
-
-
Categories
- Topics
- Hybrid IT with Cloud
- Mobile & IoT
- IT for Data & Analytics
- Transformation
- Strategy and Technology
- Products
- Cloud
- Integrated Systems
- Networking
- Servers and Operating Systems
- Services
- Storage
- Company
- Events
- Partner Solutions and Certifications
- Welcome
- Welcome
- Announcements
- Tips and Tricks
- Feedback
-
Blogs
- Alliances
- Around the Storage Block
- Behind the scenes @ Labs
- Converged Data Center Infrastructure
- Digital Transformation
- Grounded in the Cloud
- HPE Careers
- HPE Storage Tech Insiders
- Infrastructure Insights
- Inspiring Progress
- Internet of Things (IoT)
- My Learning Certification
- Networking
- OEM Solutions
- Servers: The Right Compute
- Telecom IQ
- Transforming IT
-
Quick Links
- Community
- Getting Started
- FAQ
- Ranking Overview
- Rules of Participation
- Contact
- Email us
- Tell us what you think
- Information Libraries
- Integrated Systems
- Networking
- Servers
- Storage
- Other HPE Sites
- Support Center
- Enterprise.nxt
- Marketplace
- Aruba Airheads Community
-
Categories
-
Forums
-
Blogs
-
InformationEnglish
Experts...Can Use Your Tips On Tuning Large Memory Server
SOLVED- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-14-2011 06:47 AM
01-14-2011 06:47 AM
This particular box is an rx8640 running 11.31 Itanium. It has 95.95Gb of memory, plus swap:
lvol2 = 10Gb, plus 6 more 8Gb disks for additional swap. Lots of CPU.
Running Oracle apps.
So, are there any caveats or tips anyone could share for tuning so much power. I don't want to increase a parm, just because I have so much, and create a waste of resources.
I want to be sure that this much memory, doesn't call for different methodology in tuning.
Many thanks,
Rita
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-14-2011 07:36 AM
01-14-2011 07:36 AM
SolutionThe Oracle Database on HP Integrity servers
http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA2-0547ENW.pdf
Running Oracle Database 10g or 11g on an HP-UX ccNUMA-based server
http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA2-4194ENW.pdf
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-14-2011 07:54 AM
01-14-2011 07:54 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
By "Oracle Apps" - what do you mean?
- Database?
- Fusion?
Have you any idea of what the overall memory requirements are for your "Apps"?
Do you see any "surplus" on memory?
Favourite Toy:
AMD Athlon II X6 1090T 6-core, 16GB RAM, 12TB ZFS RAIDZ-2 Storage. Linux Centos 5.6 running KVM Hypervisor. Virtual Machines: Ubuntu, Mint, Solaris 10, Windows 7 Professional, Windows XP Pro, Windows Server 2008R2, DOS 6.22, OpenFiler
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-14-2011 01:28 PM
01-14-2011 01:28 PM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Regarding "...so much power", what does "machinfo" show?
Examples:
# machinfo
CPU info:
8 s (1.6 GHz, 20 MB)
4794 MT/s bus, CPU version 4
64 logical processors (4 per socket)
Memory: 981492 MB (958.5 GB)
Firmware info:
Firmware revision: 002.074.000
FP SWA driver revision: 1.18
IPMI is supported on this system.
BMC firmware revision: 1.02
Platform info:
Model: "ia64 hp Superdome2 8s"
...
and
CPU info:
64 Intel(R) Itanium 2 9000 series processors (1.6 GHz, 18 MB)
533 MT/s bus, CPU version C2
128 logical processors (2 per socket)
Memory: 981492 MB (958.5 GB)
Firmware info:
Firmware revision: 7.44
FP SWA driver revision: 1.18
IPMI is supported on this system.
BMC firmware revision: 24.04
Platform info:
Model: "ia64 hp superdome server SD64B"
...
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-14-2011 03:38 PM
01-14-2011 03:38 PM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Kernel parameter base_pagesize = 16 will reduce the CPU overhead for managing all that memory.
We've had good experiences setting base_pagesize = 64 on our HPVM hosts (not HPVM guests) with 96GB of memory. base_pagesize = 16 seems to be optimal for our Oracle application and database environments.
As always your milage may vary.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-16-2011 07:26 PM
01-16-2011 07:26 PM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
FWIW - if it is going to be a single nPar, single Instance, Lots of Surplus mem kind of environment -- and IF the env needs superfast I/O for some routine or processing -- I'd allocate a part of Memory as "RAMDISK" -- to me that is the BEST use of memory that would otherwise just be "wasted" in cache.
Exmples of RAM DISK use would be sort areas and TEMP TABLESPACES...
Favourite Toy:
AMD Athlon II X6 1090T 6-core, 16GB RAM, 12TB ZFS RAIDZ-2 Storage. Linux Centos 5.6 running KVM Hypervisor. Virtual Machines: Ubuntu, Mint, Solaris 10, Windows 7 Professional, Windows XP Pro, Windows Server 2008R2, DOS 6.22, OpenFiler
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-18-2011 06:40 AM
01-18-2011 06:40 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Here is the machinfo on the box requested, and the box is I believe running or going to be running Oracle 11gR2.
CPU info:
7 Intel(R) Itanium 2 9000 series processors (1.6 GHz, 12 MB)
533 MT/s bus, CPU version C2
12 logical processors
Memory: 98254 MB (95.95 GB)
Firmware info:
Firmware revision: 9.022
FP SWA driver revision: 1.18
IPMI is supported on this system.
BMC firmware revision: 3.01
Platform info:
Model: "ia64 hp server rx8640"
Machine ID number: n/a
Machine serial number: n/a
OS info:
Nodename: no-name-server
Release: HP-UX B.11.31
Version: U (unlimited-user license)
Machine: ia64
ID Number: 0769464687
vmunix _release_version:
@(#) $Revision: vmunix: B.11.31_LR FLAVOR=perf
Now...what I don't know yet is if the DBA's plan to, or even know about, turn on for Oracle Optimizing using ccNUMA or NUMA.
I'm waiting to hear about that.
Zygmunt's docs were very helpful. But now I'm looking for a white paper called "Dynamic server resource allocation with Oracle Database 10g or 11g on an HP-UX ccNUMA-based server" - and so far I can't find it under the new docs webpage. So if anyone knows where to find this...let me know.
Points will be generously given later,
Many thanks!
Rita
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-18-2011 06:45 AM
01-18-2011 06:45 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
AFAIK, 10GR2 and 11G are both NUMA aware and have lots of tools to auto-this or auto-that BUT nothing beats a superfast I/O for parts of Oracle that NEED superfast I/O. Your DBA should be able to tell you which -- but TEMP Tablespace and other sort areas or other transient TableSpaces that require super high speed mosdt certinly would be very good use of that excess memory -- that is IF you forecast your 96GB of RAM will have some surplus.
Favourite Toy:
AMD Athlon II X6 1090T 6-core, 16GB RAM, 12TB ZFS RAIDZ-2 Storage. Linux Centos 5.6 running KVM Hypervisor. Virtual Machines: Ubuntu, Mint, Solaris 10, Windows 7 Professional, Windows XP Pro, Windows Server 2008R2, DOS 6.22, OpenFiler
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-18-2011 07:01 AM
01-18-2011 07:01 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
What's I'm seeing is that the server itself is not set up for NUMA (i.e. policy & LORA_MODE show default).
I'm "hoping" to get some info from DBA's or mgmt to clarify what the DBA's plans are, because if they 'think' that all they have to do is flip the switch on their side and it's a done deal, then everyone is in for a surprise.
I know if they turn on Oracle Optimization that I will need to change the settings to ensure it's mostly CLM and the rest ILM. BUT, what else do I need to do to ensure performance- what other parms might I need to tweek.
Cause once they turn it on, you can not easily change parms with CLM - not even dynamic ones with things up and running.
ILM tuning is one thing, but CLM is something I'm such a newbie on.
And I'm having trouble trying to find the white papers I want to review....wish HP would have left well enough alone. Now it's harder than ever to find things.
Thanks,
Rita
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-18-2011 05:25 PM
01-18-2011 05:25 PM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
I believe that this is the whitepaper you want:
http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA3-1350ENW.pdf
Regards!
...JRF..
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 06:05 AM
01-19-2011 06:05 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Now I have a bit of info on NUMA, and I know how to turn it on. Unfortunately, no where in anything do I recall it give any insights as to what to set some static parms, like filecache_max/min; maxdsiz; maxssiz; shmmax for few. In fact, it doesn't give any parm recommendations that I can recall from my reading the last few days. But it does say that you either go with dynamic configuration or NUMA optimization, but not both.
So...my question would now say:
If they don't go with grid technology and NUMA and they opt to stay with ILM (Interleaved memory).....are there any tips for parm setting you folks might recommend I watch out for with this box?
Thanks,
Rita
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 06:09 AM
01-19-2011 06:09 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Back online after a long break.
filecache_max/min;
set these two as you would on older HP-UX. Note that Oracle does sometimes benefit from file cache so you might find increase does help.
Oracle actually dictates some of this in the installer, and violating those rules is done at the peril of the sysadmin. shmmax needs to be equal to physical memory even though it can not possibly be that large.
maxdsiz; maxssiz; shmmax
Regards,
SEP
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 06:30 AM
01-19-2011 06:30 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
I *would* focus "tuning" and turbo-charging your Oracle DB Server environment on two things:
1) Processor binding to certain Oracle Processes
2) FastIO! (aka RAM Disk) for key
I would also advice your DBAs to go with ASM Storage -- preferably 2 or 3 ASM Diskgroups instead of using traditional RAW or CookedIO -- no matter what the overhead projector says on improvements with VxFS/DirectIO.
Favourite Toy:
AMD Athlon II X6 1090T 6-core, 16GB RAM, 12TB ZFS RAIDZ-2 Storage. Linux Centos 5.6 running KVM Hypervisor. Virtual Machines: Ubuntu, Mint, Solaris 10, Windows 7 Professional, Windows XP Pro, Windows Server 2008R2, DOS 6.22, OpenFiler
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 07:58 AM
01-19-2011 07:58 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Rita
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 08:04 AM
01-19-2011 08:04 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Pls. also be aware of:
http://skrajend.blogspot.com/2008/09/numa-after-10204-upgrade-is.html
In case you end up your DB is not using all available CPUs.... That is IF you're on 10Gr2
Favourite Toy:
AMD Athlon II X6 1090T 6-core, 16GB RAM, 12TB ZFS RAIDZ-2 Storage. Linux Centos 5.6 running KVM Hypervisor. Virtual Machines: Ubuntu, Mint, Solaris 10, Windows 7 Professional, Windows XP Pro, Windows Server 2008R2, DOS 6.22, OpenFiler
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 09:17 AM
01-19-2011 09:17 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
"I would also advice your DBAs to go with ASM Storage -- preferably 2 or 3 ASM Diskgroups instead of using traditional RAW or CookedIO -- no matter what the overhead projector says on improvements with VxFS/DirectIO."
From performance point of view the ASM is good, but not the only solution. Comparable performance can be achieved on filesystem with Oracle Disk Manager (ODM).
Read the doc:
"Improving the performance of single instance Oracle on file systems: Analyzing the impact of Oracle Disk Manager (ODM) â a feature of HP Serviceguard Storage Management Suite"
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01917294/c01917294.pdf
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 09:53 AM
01-19-2011 09:53 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
I won't disagree with ya but it is really what the DBA and SysAds are comfy about and which offers the best manageability and less risks. WIth ASM, there is no filesystem/mount point to corrupt, no "volume manager layer" to deal with. Sure with ASM a DBA has to maintain a small DB Instance -- +ASM that acts as manager of provisioned disks - but compared to traditional storage solutions - this is a whole lot simpler and moves the responsibility of manaing and monitoring DB storage to our DBA Amigos...
Say -- is ODM an evolution from Veritas Database Edition sir? Does it use Veritas bits (i.e,. FIlesystem?)
We've used Veritas DBED in the past but moved most of our DBs to ASM on HPUX (and ASMLib flavoured ASM on Linux) -- practically zilch issues compared to when we were on sashimi (raw) and sushi (cooked filesystems with DirectIO).
Cheers!
Favourite Toy:
AMD Athlon II X6 1090T 6-core, 16GB RAM, 12TB ZFS RAIDZ-2 Storage. Linux Centos 5.6 running KVM Hypervisor. Virtual Machines: Ubuntu, Mint, Solaris 10, Windows 7 Professional, Windows XP Pro, Windows Server 2008R2, DOS 6.22, OpenFiler
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 10:14 AM
01-19-2011 10:14 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Thanks for the blog pointer. It mentions his problem with only seeing 3 CPU, but it doesn't go into detail though.
For example:
On HPUX NUMA is turned off by default, and he does not mention if HPUX was properly configured for NUMA.
On the Oracle side NUMA, depending on version is controlled through an initialization parmater and only checked at startup time! On 10gR2 and 11gR1 I think it's called "_enable_NUMA_optimization". I don't think even Oracle likes this parm. Now on the 11gR2 there is a new one "_enable_NUMA_support" that Oracle supports.
So on startup, even though Oracle may have the right startup/enable NUMA parm set as "on" for the 10g and later versions - with a caveat that folks miss. Oracle NUMA will first go out and check if the server is set up for NUMA. If it is, then it proceeds with it's next steps to confirm allowing optimization. If the HPUX is NOT set (i.e. LORA_MODE=0), then Oracle NUMA is disregarded and it's optimizations will be disabled.
There's a bunch of other parms and settings that Oracle must give attention to, that clearly in his blog they obviously didn't...but what all of this has shown is that doing a ccNUMA or NUMA configuration requires DBA's & SysAdmin's to work together.
Introducing a whole new concept b/t DBA's and SysAdmins, cause Grid Technology is not for the timid.
Regards,
Rita
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-19-2011 10:29 AM
01-19-2011 10:29 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
For myself, I will try to be open to the notion of trying/testing Oracle's disk management utility....but at this point in time I can not say that I am in anyway moved to use it in any production environment.
FYI...I was actually a DBA for a very short time, and the one thing it taught me was that I knew nothing about Infrastructure. I say leave HPUX and Storage to the professionals...and unless the DBA is a certified HPUX and certfied Storage person-they shouldn't play with what they do not know.
Just my thoughts,
Rita
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
01-20-2011 05:11 AM
01-20-2011 05:11 AM
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Re: Experts...Can Use Your Tips On Tuning Large Memory Server
Was hoping to get more parm advice in the event they go with ILM and not NUMA. Although according to one document, Oracle actually has no way of knowing if the box is set up to run with NUMA or not. Now, isn't that interesting!
Guess for tuning parms, if we go ILM (native) I'll have to go by the standards I'm used to and 'tweek' from there.
Kindest regards to all and Many Thanks!
Rita
This thread is now closed.
Hewlett Packard Enterprise International
- Communities
- HPE Blogs and Forum
© Copyright 2018 Hewlett Packard Enterprise Development LP