System Speed

[Peter Clarke]

Hi,

I have a some oracle database files that were located on a four disk spripeset we have now moved some of these files onto another disk which is 3 disk raid5... now the system seems to have slowed would this have had an effect and if so how much difference is there, or should i be looking elsewhere???

[Ian Miller.]

There is some overhead in maintaining the RAID5 compared to a stripeset. How much change have you seen and are the scsi bus and disks the same ?

[Karl Rohwedder]

On your 4-disk stripeset, all 4 member are actively writing data, depending on the chunk size, the databsae IO's are spread across all disks.

On the other hand, on a 3-disk RAID5 set, for every write the according parity block has to be read, the parity recalculated and the parity block has to be rewritten ( a somewhat simplistic description). How big ths performancepenalty is, depends largely on the available cache in the raid-controller.
Without cache, a RAID5 is not good at writing.

But notice, that a 4disk stripe set has a very poor MTBF, because, if one disk fails to whole set fails. This is normally avoided by shadow the stripeset members.

mfg Kalle

[Hein van den Heuvel · Accepted solution]

As pointed out, and as you probably knew, raid-5 writes have additional IOs. For each simple (not spanning chunks) write the storage system needs to read-orig, read-parity, write-new-date, write-new-parity.
So 4 IOs for the the price of 1.
For a stripeset that is just 1 IO done for 1 Io issued.
Now the controller, thanks to write-back caching technologies, will hide the increased write latency. It will probably report near instant writes (1ms?) ... faster than a single disk IO (5ms?). However, those IOs do need to be done even if the writer is not waiting, and the disks will be activated, which can seriously disturb read IO. Readers, have no choice but to wait.

Now let's exagerate a little. Let's say your disk can do 4 IO/sec, you have 1 IO/sec write to the set and 3 IO/sec read, all random over the block space.

With the stripeset you have 4 physical IO/sec to 4 disks. That can handle 16 IO/sec. Very simplistically speaking you will have 1 in 4 odds that you have to wait for an IO in progress.
With the 3 member raid-5 you will need 7 pysical IOs and can handle 12 IO/sec. Suddenly yoy have more than 50% odds that a read will be delayed.

For a 99% read load, a raid-5 set will perform as well as a stripeset with the same amount of members for anything less than 90% read, you'll need MORE members in a raid5 set than a stripeset to be able to handle the 30% (or worse) increase in physical IOs.
You have FEWER member, so you can expect a slowdown even with modest activity.

The exact behaviour of course depends on many factors: IO controller, bus, disk speeds, read-write ratio, IO/sec MB/IO and so on.

Hope this helps some,
Hein.

[Jan van den Ende]

Hein,


So 4 IOs for the the price of 1.


I would rather define that as
You have 1 (functional) IO for the price of 4 (actual) IOs !! :-)

Proost.

Have one on me.

jpe

[John Gillings]

RAID5 was a great idea when disks were small and expensive. It provided a fairly good tradeoff - large(r) capacity virtual drives with reasonably good reliability for a moderate reduction in performance.

Today disks are large and cheap. The RAID5 tradeoff is a false economy. For a small increase in price you can RAID0+1 all volumes, or, even better host based volume shadow your stripe sets. You get excellent reliability and performance, even in failure mode.

Even worse... We find people think RAID5 volumes made from modern high capacity drives are "too big" so they partition them. Contention between the partitions makes performance terrible (worst possible seek pattern), and, if a physical disk ever fails, performance is really atrocious.

RAID5 is a technology that's had its day. Please try and avoid it!

[Tom O'Toole]

Amen to John,

raid5 leaves you very vulnerable after you've had a failure, while the entire storage set rebuilds to include a spare drive. A failure during this time = you're toast. If you then want to replace the fialed drive in the raid set, you remove the spare, and add the replacement, creating another window of vulnerability.