Disk Enclosures
1748065 Members
5451 Online
108758 Solutions
New Discussion

Linux cciss driver with Smart Array 6404 / 6402 RAID6 slow performance

 
Daniel Pocock
Occasional Advisor

Linux cciss driver with Smart Array 6404 / 6402 RAID6 slow performance


I've been using the cciss drivers with various generations of HP array controllers and drive cages. I've generally found that they work quite well with Linux, and are not particularly difficult to install and configure.

Nonetheless, I'm now seeing severe performance problems with some large arrays (both U320 and SATA), and the cciss documentation doesn't even suggest where to start troubleshooting this issue.

Hardware:
- server: DL585

- controller: Smart Array 6404 in PCI-X 133 slot
- controller: P600, PCI-X 133 slot

- drive cages: 2x MSA30, connected to channels A1 and A2. Channel B1 and B2 not used
- drive cage: StorageWorks 60, SATA drives

- drives: 28x 300GB 10k U320
- drives: 12x 750GB SATA

Both RAID controllers have the latest firmware.

The drives are arranged in RAID6:
- 28x 300GB drives = 7.0TB RAID6, four logical volumes
- 12x 750GB drives = 7.5TB RAID6, one logical volume

Doing read and write tests with dd and iostat, no other activity on the arrays:

6404 and U320 drives:
- read 130MB/s
- write 75MB/s

P600 and SATA drives:
- read 155MB/s
- write 20MB/s

These performance figures are not too good - we have another identical server with a Smart Array 641 controller, and 14 drives RAID 1+0:
- read 60MB/s
- write 110MB/s

Does anyone have any suggestions for troubleshooting and performance tuning with the cciss driver, the various Smart Array controllers?

The tests were done with 100GB files.

I've confirmed that write caches are enabled.

The RAID6 U320 logical volumes were only created an hour before running the tests - is it possible that the controller is still doing some kind of array setup work in the background?
1 REPLY 1
gatuk
Occasional Contributor

Re: Linux cciss driver with Smart Array 6404 / 6402 RAID6 slow performance

This is an old thread and I don't have a proper solution for you but I too have seen this across a wide range of HP controllers and numerous versions of Linux over the past few years.

I have worked around it normally by exporting the raw disks and doing all the RAID in software. This gets the performance closer to what you might expect - in most cases.