HP P212 RAID-Controller is extremely slow (having a Simple RAID-1-Array)

[Thomas Wunder]

Hi,
having bought a new Proliant DL120 G6 (Performance) i've set up a simple RAID1 Array with two brand new Western Digital WD20EARS (SATA300, 2TB, 3,5", 7200U/m, 64MB) and tried to install Debian Squeeze (AMD64). The Debian Installer seems to recognize the Array correctly but it takes round about 1,3 hours to complete. The installed system isn't even faster. I ran a

hdparm -t /dev/cciss/c0d0

and it reported a read throughput of 73 to 85MB/s which isn't even as fast as my desktop PC which has only one simple Disk (non-Raid) installed. I've also tried to run the same hdparm test from a Gentoo-Live boot and it reported approximately the same values.

(By the way: 'lspci' reports the device to be a
"RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01)")

I've also installed cpqacuxe and inspected the Controller/Array/Disks. No error or warning was reported. And there's virtually no message in the system log that could hint at a driver/system/kernl/filesystem/hardware failure.

Someone told me that the controller might be syncing the disks but i had it resting idle for about 8 hours and it's still as slow (shouldn't the sync be finished in the meantime?). The other thing is that is absolutely no harddisk activity (i.e. the green LEDs are off as long as no filesystem from that array is mounted).
How can I find out whether it is currently syncing?

I'm really desperate and I hope that you can give me some hints how to resolve that problem...


Thanks in advance!
Tom

[Johan Guldmyr]

Hi!

Did you get the p212 with a cache/extra memory attached to it?

You can try to install "array diagnostic utility" on the server and see if that tells you anything regarding the status of the controller/disks maybe.

DL120 G6 has intel CPUs inside.

I don't knowenough about Linuxbut should you use the x64 version or whatever it might be called of Debian then?

[Steve Reece_3]

I'm not sure of the config of the P212 controllers, but my experience with the RAID controllers on the rx2660 is that if you have a controller without battery-backed write cache, performance can be extremely slow. This I guess is due to needing to write to the rotating media before the write is indicated as being committed rather than just writing to cache and moving on.

If you've got a controller without BBWC, I'd see what you could do to upgrade it with BBWC.

Steve

[Thomas Wunder]

Hey Guys,
i guess i know the reason of the problem now. It's because of the harddrives. I'm using the Western Digital Caviar Green series WD20EARS disks which use the "Advanced Format" i.e. they have a block-size of 4096 bytes. The problem is that they either don't send the correct block-size value or the controller doesn't detect it properly (as the CPQACUXE web-tool reports it to be 512 bytes of size).

I did the following to check this hypothesis and i guess i can accept it as proven (running a Gentoo Live USB-Image):


livecd ~ # time dd if=/dev/zero of=/dev/cciss/c0d0 bs=512 count=4096000
4096000+0 records in
4096000+0 records out
2097152000 bytes (2.1 GB) copied, 69.499 s, 30.2 MB/s

real 1m9.500s
user 0m0.720s
sys 0m8.030s

livecd ~ # time dd if=/dev/zero of=/dev/cciss/c0d0 bs=4096 count=512000
512000+0 records in
512000+0 records out
2097152000 bytes (2.1 GB) copied, 24.694 s, 84.9 MB/s

real 0m24.695s
user 0m0.030s
sys 0m1.220s


Now the question is whether there is an option to override the block-size value which is provided by the controller and set it manually/explicitly to 4k. Any suggestions?

[Patrick Terlisten]

Hello Tom,

skip the idea with the block size. For backward compatibility the disks reports not one 4096 byte sector, but eight 512 byte sectors. There is no need to "configure" the controller for 4K sectors.

Regards,
Patrick

[Patrick Terlisten]

It's me again. You test with DD is useless. Of course writing/ reading with bigger IOs is faster then doing the same with smaller IOs. Using bigger block sizes with DD will result in more MB/s.

Regards,
Patrick

[Thomas Wunder]

Hi Patrick,
I can't agree with that completely. I've also tried the "dd-test" with block sizes of 1024, 2048, 8192 and even with "odd" values like 4099 etc. It became evident that the high transfer rates of about 80 to 90MB/s could only be achieved when the block size which was used was a multiple of 4096. All the other values like 2048 and the odd values resulted in transfer rates of about 30MB/s.

To say it once again: I don't think that this issue is mainly caused by the controller but by the disks. The problem is simply that the OS would not cope with a wrongly reported block-size (at least it is a bit complicated to set up partitions and filesystems manually to respect the 4k setting...)

Thanks anyways!
(By the way what would you suggest to get it working more efficiently?)

[gregersenj]

If you want more speed, you will need faster disks.

I'm not a shark on performance.
But 80-90 mb/sec seem Ok to me, when you are copying 2,1 GB

the systained transfer rate of the disk is 110 MB/Sec
http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-701229.pdf

There's no way a RAID controller can make a disk faster than it is.

BR
/jag

[Thomas Wunder]

Hi gregersenj,
I've already solved the problem and now I'm really pleased with the performance.

The solution consists of multiple things:
1. I needed to activate the write cache of the controller by enabling the 'Override Write Cache Battery' setting.
2. I've set up the partitioning with some special settings (256 heads, 56 sectors/track which seems to have caused another drastical performance improvement. The other thing it has caused is that the default block-size for the ext3 filesystem inside the partitions is now 4k.

I've spent about 3 days to find this solution and I've tried virtually everything and every possible setting. As of now I'm pretty happy with the performance (ca. 105MB/s) though these things only hold true for my WD20EARS disks which are 4k disks. I can't really explain why it works now and another guy from this board (Patrick Terlisten) insists that step 2 of my solution is not necessary or even disadvantageous but as stated earlier it works and it's fast - much faster than before.

Everybody who doesn't believe it is invited to come and see/try it.

The only problem still have is that it doesn't boot from that array any more though it did before. I still suspect this to be caused by the firmare update I did while I tried to get more performance but I'm absolutely unsure about this.
There's more information about that new issue here:

http://forums13.itrc.hp.com/service/forums/questionanswer.do?threadId=1431782

[Pieter 't Hart]

You can find more posts on the forum about SATA disks behind a smartarray raid-controller, where the cache on the drive "interferes" with the caching on the controller.

Try to disable caching on the disks (sorry, cant tell how) and verify that you have write-back cache enabled on the raid controler. (acu should be able to give this info).

[gregersenj]

105 MB/sec is very fine :)

This advisory is old, but it may still apply.
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01209560〈=en&cc=us&taskId=120&prodSeriesId=3745263&prodTypeId=329290.

I have no doubts, that fine tuning the OS to match the RAID controller improves perf.

But what I wanted to state is, don't expect more than the disks can deliver.

BR
/jag