- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Oracle ASM - more exacting, load on IO?
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-20-2009 04:23 PM
тАО01-20-2009 04:23 PM
Before, Our Filesystems (24 total) used to be on 8-way stripes (8 physical/SAN disks per).
On our new ASM Layout, we provide our DBAs with 48 disks. ASM suppsoedly stripes accross all these 48 disks.
Sar and Disk stats indeed point a uniform load accross these 48 disks but my service times jumped from sub-20ms to ~130ms. And my Array front end now show its port processors are doubly busy than before.
So is ASM more heavy, exacting on Storage Arrays? That it merits a relayout frontend. channel and even array group wise on the SAN Array end?
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-20-2009 05:24 PM
тАО01-20-2009 05:24 PM
Re: Oracle ASM - more exacting, load on IO?
I've seen these kind of results from ASM in the lab as well. ASM acts like it owns the disks, knows better, and takes over certain functions normally done by the OS.
ASM exacts a higher price an any kind of disk you give it. I believe the SAN/Disk array configuration is the better place to control disk, and am not a believer in ASM as a solution.
SEP
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-20-2009 09:00 PM
тАО01-20-2009 09:00 PM
Re: Oracle ASM - more exacting, load on IO?
The file system may have been helping more than realized.. by using memory as cache.
In other words.. do you see less memory used in glance/vmstat since the switch?
Consider giving that now unused memory directly to oracle (SGA)?
20ms is not great.
130 ms is pretty darn slow.
Do you get user complaint now?
Is that response time measured with iostat (averaged over all disks)? Maybe this is mostly cause for example by spikes writing to archive logs which the end users do not feel? Time for redo and arch log (writer/buffer) tuning?
Can you check current and historical performance reports (statspack, awr) to figure out how Oracle perceives the IO response times? Does it 'add up'? Check read vs write, check whether particular object have vastly different Io response times.
Good luck,
Hein van den Heuvel
HvdH Performance Consulting.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-20-2009 09:05 PM
тАО01-20-2009 09:05 PM
Re: Oracle ASM - more exacting, load on IO?
Just think of ASM as a filesystem for database stuff. That is, at the core, what it is.
A special purpose filesystem.
ASM was designed to take all of the devices and stripe over all of them.
here you have the CHOICE of redundancy: either you use ASM's mirror and stripe everything approach or external hardware redundancy.
So it you are providing your DBA with 48 disks (which is already in a RAID), you should also inform the DBA NOT to define any further level of redundancy.
can you confirm your configuration?
(RDBMS Instance: SELECT * FROM V$ASM_DISKGROUP;)
kind regards
yogeeraj
Ps. also read: http://download-west.oracle.com/docs/cd/B19306_01/server.102/b14220/mgmt_db.htm#DCGGGJC
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-21-2009 12:31 AM
тАО01-21-2009 12:31 AM
Re: Oracle ASM - more exacting, load on IO?
I have to say my first thought was also that the DBAs still had redundancy enabled in ASM (effectively doubling the IO rate).
So your service times have increased by a factor of 6, what about your throughput (MB/s) and IOs per second? Do you have any stats for changes there? (e.g. from Measureware looking at GBL_DISK_PHYS_BYTE_RATE and GBL_DISK_PHYS_RATE)
Also what about MPIO - how were you doing MPIO before and how are you doing it now? Any changes (cos ASM of course does no MPIO itself).
HTH
Duncan
I am an HPE Employee
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-21-2009 12:19 PM
тАО01-21-2009 12:19 PM
Re: Oracle ASM - more exacting, load on IO?
I am still in the process of gathering stats from my XP12000, System Ststs (SAR and GlancePlus/OVPA) -- but it really appears we're over the top and those service times are worrisome.
And yes my DBA says he uses external redudancy in ASM:
SQL> @$ORACLE_BASE/local/asm/asm_diskgroups
Disk Group Sector Block Allocation
Name Size Size Unit Size State Type Total Size (MB) Used Size (MB) Pct. Used
-------------------- ------- ------- ------------ ----------- ------ --------------- -------------- ---------
PRDB001_DATA 1,024 4,096 1,048,576 MOUNTED EXTERN 2,555,376 2,531,335 99.06
--------------- --------------
Grand Total: 2,555,376 2,531,335
We are using VxVM underneath each component ASM "volume" -- which is a VxVM Volume that is wholly contained on one XP12K Disk. I can monitor the entire member disks using vxtools and I see ALL 48 member volumes active ALL the time as if data is striped 48-way (which is suppsoed to be how ASM works). I can validate the cXtYdZ dveices of these volumes via SAR and the service times for reads and writes are indeed breaching 100+ ms service times.
Now compare that to our OLD cooked Filesystems which were 8-way stripes per Filesystem/Volume. The database hardly stress HALF of however many disks we had as I.O jumps (interleaves) from FS to FS. But with ASM -- it seems ALL 48 disks are always busy (striped) that I am noticing my XP12000 front end processors to be far more utilized/sterssed than we were on Cooked or even simple RAW (with 8-way stripes)..
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-21-2009 12:22 PM
тАО01-21-2009 12:22 PM
Re: Oracle ASM - more exacting, load on IO?
Do you agree?
I am still poring through stats though.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-21-2009 12:53 PM
тАО01-21-2009 12:53 PM
SolutionWhat sort of IO is the DB doing? Are there a lot of full table scans generating sequential IO? Doing that over 48 disks on multiple ports might make that sequential IO harder for the XP to spot (confusing the cache algorithms) I'd be interested in trying a test with your DMP turned off as well (i.e. all IO to just a primary path)
HTH
Duncan
I am an HPE Employee
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-21-2009 02:20 PM
тАО01-21-2009 02:20 PM
Re: Oracle ASM - more exacting, load on IO?
But do you agree with my assumptions? I will hopefully soon have stats to support this claim.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-22-2009 12:32 AM
тАО01-22-2009 12:32 AM
Re: Oracle ASM - more exacting, load on IO?
I'm not sure I *do* agree... is the database doing anything different? Is the DBA seeing much improved performance/throughput? The total amount of IO generated regardless of filesystems, raw or ASM shouldn't be significantly different (I guess the number of IOPS could change if ASM is reading/writing with different block sizes than VxFS was? In fact that gets me thinking... maybe with VxVM in the equation the ASM IO block sizes for reads/writes aren't matching the VxVM IO raw disk block sizes for reads writes? You'd know this if your total IOPS have gone up, but your throughput in MB/s has stayed largely the same. IIRC ASM will simply read in the sizes determined by the database parameters (i.e. database block size * multi-block read count) - how does that comnpare to the sizes that VxVM is reading?)
Also are you sure that ASM isn't doing any rebalancing? It will do thgis automatically if disks are added to an ASM disk group. Does the table V$ASM_OPERATION show anr jobs running? That would generate significant additional IO.
Finally waht about the internals of the XP? When you were on VxVM with 8 way stripes, you probvably had it configured so that each LUN in a strip came from a different array group and maybe even a different DKA/ACP... now you may have LUNs from the same array group in the same disk groupo - again if you have a lot of sequential IO this could cause some challenges as the disk heads thrash back and forth between the different ASM stripes on the same physical disks (if you follow me)
Anyway some things to think about. It'll be interesting to see those IO numbers.
HTH
Duncan
I am an HPE Employee