RAID 0+1 performance with different number of drives

[Michael Neymit]

Recently I conducted several experiments to compare performance of RAID0+1 with performance of RAID5. The tests I conducted included copying files to and from the drive arrays. I also built several test databases on various types of arrays and performed large volume database manipulations, while timing their duration. As expected RAID 0+1 consistently outperformed RAID5 (by about 30-40%). What puzzled me is that I didn't see any difference between different configurations of RAID 0+1. For example RAID 0+1 consisting of just 4 drives (bare minimum) had the same performance as RAID 0+1 comprised of 12 drives. I did multiples tests and the results were consistent throughout. This tells me that there must be some other bottleneck, but what? My configuration:
Server -- Compaq ProLiant 8500
CPU -- 4x550MHz
RAM -- 4GB RAM
Controller -- 4200
Drives -- 18.2GB Ultra SCSI 2, 10K rpm

Any thoughts?

Thank you!

[Eugeny Brychkov]

The advantage of RAID 0/1 is that data can be read and written to striped drives in parallel. In your case, RAID 0/1 looks like operates like RAID 1, reading/writing only from one pair of drive at a time. I'm not familiar with your RAID array, but would think that you may have option to setup such array behavior in its management utility or bios. Maybe firmware update will help?
Eugeny

[Brent Boisvert]

Try increasing the amount of controller cache or changing the caching preferences from the default of 50%-read/50%-write to suit your environment. Remember, RAID0 is read once, write once. RAID1 is read once, write twice. RAID0+1 follows the same rules as RAID1.

Regards,
Brent

[Vincent Farrugia]

Hello,

I think performance remained the same because you have only 2 controllers.

Usually, the best method for performance is to stripe or mirror across controllers. If you have 2 controllers, you can stripe the mirrored disks across one another. So if you have 4, 6, 8, 10, 12 etc., you still are striping across two controllers, so performance gain will be minimal, if at all.

HTH,
Vince

[Michael Neymit]

I think Eugeny is onto something. It really does look like my RAID 0+1 behaves like RAID1. As for the caching options, I would think it affects all the configurations equally (as long as they are attached to the same controller). Again, the point I was trying to make is that with RAID 0+1 consisting of just 4 drives, I effectively have a sripe with only 2 drives (and the other two are the mirror). With RAID 0+1 composed of 12 drives, my stripe is now 6 drives. Wouldn't you think that read/write to 6 drives in parallel should be faster then read/write to 2 drives in parallel?

Thank you for your replies!

[Eugeny Brychkov]

Modern SCSI devices should support SCSI select and reselect commands. This means that SCSI controller selects target, sends request to target, then deselects it. During target's request processing controller can select another device and send request to that device. As soon as first target is ready with data controller reselects it and reads data.
It can appear that bus got locked until device becomes ready with data, so controller simpy stands by.
This is brief explanation. See difference? So maybe you need to dig into RAID controller setup to find an option to change
Eugeny

[Brent Boisvert · Accepted solution]

More spindles usually means better performance as long as each spindle in the RAIDset is on a different SCSI bus or channel. Once the RAIDset "doubles back" or has more than one member on the same SCSI bus, performance will flatten out. For maximum performance, do not include more members in your RAID set than your controller has channels. (I realize that this is usually not feasable, but I'm being theoretic.)

Brent

[Michael Neymit]

If what Brent is saying is correct, it would completely explain what is going on. Can anybody confirm or deny his statement?

Thank you!

P.S. I am not a hardware expert, but a Database Administrator, so all replies are very much appreciated.

[Leif Halvarsson_2]

Hi,
What kind of performance have you measured. If you talk about transfer rate (for example when copying large files) you will perhaps not get better performance with 12 drives than with 4 (SCSI-bus bottleneck).

When you tested with copying files, what vas the average filesizes ?.

Is it possible for you to do 2 tests, one with small files (10k) and one with rather large files (1M), do you get any difference ?

I am not sure Brent is correct, of course it is better with more disk channels but 1 per drive ?. Most Raid controllers have only 2 or 4 disk channels.

I am working with a reconfiguration on one of our Raid systems, if there is time I will do some tests myself.

[Vincent Fleming]

It's unlikely that you are exceeding the bandwidth of your back-end SCSI busses with 2 drives on the bus - or even 4 drives.

Your back-end SCSI buses are good for up to about 5000 IOPS or 80MB/s, right? A single disk drive (10K) is good for about 150 IOPS or about 20-30MB/s. And that's read rates - write rates of drives are significantly lower.

So, you should be able to get several drives on each bus before you are loading it heavily.

You don't say what your performance is on the RAID 1/0 - just that it's higher than the RAID5. You could be hitting a bottleneck somewhere such as the PCI bus (about 264 MB/s max) or within the RAID controller.

Now, you say you're testing RAID 0/1 vs RAID 5... you should note that not all RAID 5's are the same. Some controllers do it better than others. HP has systems where the RAID 5 performance is much closer to RAID 0/1 performance... particularly with sequential loads and read-intensive loads, where RAID 5 can outperform RAID 0/1.

Anyway, good luck!

Vince

[Brent Boisvert]

I realize that placing only one disk per channel is extreme and not practical.

Consider a three disk RAID5 on a two-channel controller. Channel A will have two drives on it and that SCSI bus will be twice as busy as Channel B with only one drive. Channel A will have a higher bus latency. This is why controllers with more channels outperform controllers with less channels. This performance hit may be very, very small, but if the server is passing many I/O intensive requests to the disk channel (writes in particular), it could multiply the latency.

Anyway, I don't believe that with the hardware stated in the original post that you should expect to see a performance gain using more drives in RAID0+1. I agree with Leif that file size may also factor here. One large file may yeild better results that many smaller files (sequential disk access .vs random disk access)

I hope this discussion has been helpful.

Brent

[Michael Neymit]

First of all thank you everybody for your replies!

Let me try to answer some of the questions above. In the tests I conducted I was copying files of various sizes, but all of them were fairly large: 7GB, 29GB, 3GB. Based on the times recorded, I calculated the average throughput (FileSize/Seconds). I found that on RAID5 the rate of file copying was 7.5MB/sec. On RAID0+1 it was 11.8 MB/sec. Not anywhere close to the limitations of SCSI bus or disk I/O, but what is the limitation of RAID controller per channel? Unfortunatly I can not provide any more details about the RAID system I used other then what was stated in the original post -- it is 4200 Compaq controller with the default settings for both RAID5 and RAID0+1.

Thank you!