Control over IO size?

[Martin Robinson]

We have SAP, Oracle, EMC. Trying to optimise performance, we have disk queues and slow response times on some disks, so are considering striping.

However the specific question concerns IO size. sar -d is reporting that our IOs are normally about 16k. I assume the man page of 11.11i is correct, that the blks/s field is in units of 512 bytes. For example, r+w/s = 100, blks/s = 3200 means average IO of 16k.

These sizes strike me as too small. Oracle is configured with blocksize=8192, db_file_multiblock_read_count=8, which means I think Oracle should be trying to do 64k IOs wherever possible.

The vxtunefs parameters are default, so read_pref_io is already 64k. Why does it look like we are getting 16k IOs?

If I am right, will we benefit from fiddling with vxtunefs parameters? Will bigger IO result in reduced avwait?

Now for the really tricky question. What stripe size should we use? Should we attempt to increase the IO size before applying striping?

Sorry, lots of questions.

By the way, we are not using raw disk for Oracle.

[Stefan Farrelly · Accepted solution]

In my experience mucking around with vxtunefs doesnt really help much so not worth the effort. To find the ideal stripe size you need to test it - try from 16/32/64/128 and test something a huge oracle report (just using dd to test it is not enough - you need to test your application/Oracle). We run tons of Oracle here and its all striped now, and we found the best stripe size = 64k. Its a huge increase in performance over non striped lvols.

[Michael Steele_2]

I/O bottlenecks often are corrected by adding more physical memory. There evaluation is more about processes hitting the disks and file systems associated to the disks. This is the per process transfer rate.

One dedicated process, like oracle, on a file system verses many processes all competing for write access.

Oracle has their own file system block that tends to be larger because in a dedicated process environment a large block size increases performance.

With many processes a smaller file system block size increases performance.

So go real large for one process on a file system but go small (* to smaller *) for many processes on a file system.

Other variables include adding more disks, this is the spindle rule and where striping would fit in.

'Striping' is load balancing and since EMC has load balancing software be sure to check with EMC before you start striping, which is a LVM procedure. But if you do stripe with LVM, then investigate it thoroughly because there are lots of tricks to it. I.e., can extend the file system easy, better to use physical volume groups, round robin PV links, etc.

(* I don't think most EMC users opt for this alternative. *)

Regarding "...what stripe size..." the default is 32K, 64k is also common, 1024k has also been making the rounds.