I/O questions

[K.C. Chan]

All,
I am in the process of laying out filesystem for an OLTP oracle Database. To make the long story short I ended up creating 2 raids on DS2300 running half bus mode. In an ideal situation, I would want raid 1 + 0. But in this situation we opt. to go with raid-5+1. Characterist of the both array is:
PowerFail Safeguard = ENABLED
Disk Spin Up = AUTOMATIC
Rebuild Rate = 30%
Firmware Version = U.01.06
DRAM Size = 128 MB
Device File = /dev/iop0
Hardware Path = 0/10/0/1
Adapter Type = 467
Speaker = DISABLED
Battery Pack = CONNECTED
Battery Status = CHARGING COMPLETED
Battery Temperature = GOOD
Battery Voltage = GOOD
Battery Charge Counter = 0
Configuration Saves = 11
Firmware Downloads = 0
Initiator ID = 7

RAID LEVEL = 5
Size = 210018MB (205.1GB)
StripeSize = 64KB
Write Policy = WRTHRU
Read Policy = ADAPTIVE
Cache Policy = Cached IO
#Stripes = 4
State = DEGRADED
Spanned = NO
#Spans = 1
Disks in Span 0 :
Ch ID Type Capacity Vendor/Product/Revision Status

0 4 DISK 70005 MB HP 73.4G ST373453LC HPC3 REBUILD
0 1 DISK 70005 MB HP 73.4G ST373453LC HPC3 ONLINE
0 2 DISK 70005 MB HP 73.4G ST373453LC HPC3 ONLINE
0 3 DISK 70005 MB HP 73.4G ST373453LC HPC3 ONLINE

Note the stripesize is 64k. On top of the raid, I stripe my lvol across the two raid-5 (BCC left and BCC right). This way I am utilizing both controller ( I know it's not enough, but it's what I got to work with). My questions is when I stripe the lvol, I use 8k as the stripe size. What is the relation of this stripe size vs. the raid stripe size? What affect does it have on each other? Thanks.

[Hein van den Heuvel]

Imho an 8kb stripe is ludicrous, and 64k rarely worth it. You should think in 128kb or bigger terms, maybe even 1 or 4 mb.

I _used_ to think 8kb stripes at controller level was interesting with scsi-1 at 5mb/sec in order to get the data in more quickly in parallel.

But now interconnects are running at 50mb/sec or better and the transfer time is largely irrelevant. Add to that the cost to parcel up an IO and you loose.

You'll get randomized/spread IO with just about any stripe size and to get any throughput you want the system to be able to put large IO requests on the bus to the controller.

Books have been written on this subject. Read them, interpret them. Don't take the word of a random person replying here for it.

Hein.

[Tim D Fulford]

Hi

I like Hien's closing comment...

I actually think going for raid5 on an OLTP will make the whole system disk limited. raid1+0 is the way to go. That said "ya got ta work with wot ya got"

Any way just to see if I understand you. You plan to do a 8k LVM stripe (over two RAID 5 LUNS), which are themselves striped at 64k.

This will mean that the disk subsystem will be HAMMERED (IMHO). Let me explain... Every time you write 8k of data the WHOLE RAID5 LUN will be read & 64k parity calculated & then 64k+8k data written back. In your case 8k write will mean 192k read & 72k write (3% efficient!!, this is why raid 1+0 is better as it is 50% efficient as far as RANDOM IO's go, sequential is another story). I think you really should invert the striping such that one LVM write would be equal to one raid5 lun stripe. Unfortunately you can't do this as LVM has a limit of 64k stripe. So the next best thing is to make LVM stripe match raid5 stripe, so then you are less inefficient (20% efficien). Alternatively... do away with kB striping & go for extent based stripe (4MB default).

my 0.02??? worth

Tim