Horrible Write Performance with Smartarray P400 in DL320, DL320s and DL1xx

[Ulrik Holmén]

Oh crap...

We bought another server (DL180 G5) with SATA and P400 (512Mb BBWC) and we have the same problem again.

The previous server the DL320S is still running and performing, though we still need to have the drives write cache enabled. This trick doesn't make it for the DL180 and this time I don't have time to go through the same troubleshooting again. This time HP will have to fix it for me.

[Chirs Werfel]

Ulrik,

I have noticed that I have had to reboot after changing the cache setting AND it often took several hours or a day for write performance to improve. I think this is based on how the controller performs parity and optimization operations.

Chris

[ldoodle]

Just got one of these cards WITH 128MB BBWC, physical write-cache enabled and accelerator enabled for both arrays.

Interestingly though, Windows Server 2008 won't let me turn on it's own write-cache - I tick the box and apply, then go back in and it's de-ticked!!!

Anyway, have 2 80GB SATA's in RAID 1 for OS, and 4 250GB SATA's in RAID 5 for data. I also have a 500GB USB drive plugged in for backups.

When writing to the RAID 5 array, write speeds are ~20MB, but when copying from the RAID 5 array to the USB drive, speeds peak at about 170MB - 8.5 faster!!

What's wrong with the write speeds on this card?

[PhilipHarvey]

We have six HP DL380 G5 servers with P400 controllers with 256MB of non-battery backed cache. Each server has 2x SAS disks in RAID1 and 4x SAS disks in RAID5.

The read and write performance on RAID1 is good. About 30MB/sec write and about 35MB/sec read.
The Read performace from RAID5 is OK but lower than you would expect, about 25MB/sec.
The Write performance for RAID5 is crap, totally unusable, worse than consumer grade junk, 10MB/sec.

I am going to test if the 5.22 firmware makes any difference, and also test if installing and enabling a battery backed controller helps and will post back the results.
It looks like a firmware/hardware fault, and if firmware doesn't fix this then I will be sending them back to HP.

[George Mason (Xoom)]

@Ste B

Hi Ste,

As I remember the driver logged an entry in the Windows Application log (or was that System log? probably...) saying that the change I had made was being disabled by the driver.

Just had a look on the server now and the difference between the earlier and later version of driver and firmware are very apparent from within the ACU. I'm running ACU v8.0.14.0 if I click on the E200 controller, then Advanced Settings (read: ESSENTIAL SETTINGS!) there's an option labelled Physical Drive Write Cache Settings from which you can enable the disks' caches.

Once i had done that, the machine worked as it should have out of the box. Never mind it nearly lost me a customer and all that....

Good luck.

George

[George Mason (Xoom)]

@Philip

I've not been able to use one of these servers for anything even remotely IO intensive without BBWC.

In our case we found BBWC made a big difference on the SAS machines. On the SATA machines we also needed to enable physical write cache as mentioned in my last post.

Good luck.

George

[PhilipHarvey]

I opened an incidnet with HP and they pointed me to a web page on their site stating that this is a know fault with the P400 raid controller, that RAID5 writes and writes to vmfs are both VERY slow and they have no intention to fix it.

The suggested work around from HP is to purhcase a battery backed cache module.

I tested using
time dd if=/dev/zero of=testfile bs=1M count=1024

On RAID5 vmfs this took 4m36s with no battery installed. As soon as I installed the battery it took 9s.

On RAID1 vmfs this took 2m00s with no battery. With battery took 10s.

On a Linux file system on a RAID5 disk the above command took 50s. With the battery installed it took 16s.

Basically you can not use the P400 RAID controller for VMWare or for RAID5 without the battery installed.

HP page on this is here:
http://h10025.www1.hp.com/ewfrf/wc/document?docname=c01670205&cc=my&dlc=en&lc=en&jumpid=reg_R1002_MYEN

[PhilipHarvey]

PS- you have to re-configure the P400 so that it uses some cache for writes before you will see performance improvement.

You can do this by booting from the Smartstart CD and using the ACU.

[George Mason (Xoom)]

Understood. This must be why most Top Value machines (DL360, DL380) are now and have been for some time shipping with BBWC included.

Glad you got it sorted.

[leisan]

Hi from Spain, I have to a P400 controler and I have the same performance problems and I have a few help, sorry for my bad english.

I read the post and I understood that the problem is that the optión "write cache" is disabled for defect.

My firmware os controller is 5.22 but I don't know where I can change the value of "write cache" to enable. The bios os controller only let me to change or view the diferents levels or RAID but it doesn't have any options. Where and how I can change the other "EXTRA" options, I need another CD or media? Thanks a lot.