AIO 600 performance

[Andrew S Willy]

Hello all,

Our AIO 600 configuration is 5 x 7.2k SATA drives in RAID 5, 2 GB RAM, Windows 2003.

At network transfer speeds of > 200 Mbps (receive/write), the AIO performance is considerably reduced. Console user's experience is sluggish. VMWare Guest Machines with iSCSI storage provided by the AIO suffer. At > 400 Mbps, the AIO becomes unusable from a console and iSCSI perspective. This occurs with NFS and CIFS shares.

We observe that System Cache during these periods uses 3/4 of the available RAM.

Is it possible that our drives are a bottleneck, and that the AIO cannot write as fast as the NIC(s) receive, causing the cache to use up RAM, eventually leading to performance degradation?

Are there papers that explain performance expectation of an AIO600 in our configuration? We have DL380s with attached MSAs that move data at similar speeds without the same problems.

Thanks in advance,

Andrew

[Eli_Dezman]

Probably you have a problem with SATA drivers .
they are very slow , use 15K drivers for VMWare and other IOPS demanding application like Sql and Exchange

[Patrick Terlisten]

Hello,

you can check with perfmon if the disks are the problem. You shouldn't have more then 10 - 15 IOs in the disk queue, and the command response time should not exceed 30ms. By the way: RAID 5 is not optimal for your enviroment, you should use RAID 1+0 instead.

Best regards,
Patrick

[roberttco]

What version of virmware are you using on the SmartArray controller - I assume you are using the E200i. The latest firmware only improves performance for LUNS over 2TB. You did not say what size disks you are using. SAS disks will provide a much greater performance point as the E200 does not support NCQ. In my tests using a P400 (not officially suported in an AIO600 - you will not be able to quick restore the system) with 4.12 F/W provides about the same throughput as SAS since it supports NCQ. The other thing to sidder is that RAID 5 exacts a huge penalty when your workload is several small writes for an extended period of time as it requires a read-modify-write cycle on every block of data that is changed when the cache is full. You might considder changing the cache configuration to more suit your workload.

[Andrew S Willy]

Thanks for the replies.

Eli - interestingly, I hadn't noticed this before, our disk drivers are listed as "HP LOGICAL VOLUME SCSI Disk Device". This system was built with the shipped with CD. Are these the wrong drivers?

Patrick - wow! If we're supposed to be at 10-15 IOs and response time under 30ms, the disks are maybe having trouble. Our %disk time avg during a medium utilizatin period (50-70 Mbps) is 112 (I assume they're described in milliseconds in perfmon? %Disk Write Time avg is 54. Avg Disk Read Queue Length is 1.7, however, the it is graphed as if it is in seconds rather than miliseconds. I'm not sure how to interpret that number. Same for Avg Write queue at 2.34.

Robert - I will get back to you with the firmware info.

Thanks to all.

[Andrew S Willy]

We are using the Smart Array E200i controller, firmware version 1.20.

[Andrew S Willy]

Our drives are 500 GB. One partition is 2.2 TB, dynamic, spanned across two 1.1 TB logicals.

[roberttco · Accepted solution]

Thanks for the information. It looks like the P800 is a supported controller for the AIO600 (P/N 381513-B21) and the QuickRestore should work fine. As far as how you have your disks configured, I assume that you need the 2TB of space to create a 2TB iSCSI LUN ? In any event there are a couple problems with dynamic disks. First there is a memory leak in the Microsoft provided dynamic disk VDS provider that is exposed while the Application Storage Manager is running on the systems - in short Dynamic disks are expressly unsupported on AIOs. Secondly, spanning two RAID5 logical disks effectively negates the fault tollerance that you have setup. Unless you can guarentee that no disk I/O ocurrs across the span boundary you might experience data corruption - particularly if a drive fails. I do not know how you have disks configured but I suspect that the two logical disks reside on the same physical drives so if you encountered two drive failures, all of your data would be lost - not just half of the spanned disk. I would be happy to discuss your options further on the phone if you would like. If you would like to do this, please call HP support and then post the call number to this forum so that I can lookup your contact information to contact you.

[Andrew S Willy]

If I have two logical drives on the same physical disks, I would lose all of my data if two drives fail whether or not my logicals are spanned, wouldn't I?

We aren't committed to disk spanning, in fact I wish I hadn't done it to begin with, however moving off 1+ TB to break the span and start over with basic disks would be a large undertaking -- especially since our AIO is performing so poorly.

I'm on with HP support now to get a case number. Will post it when I get it.

[Andrew S Willy]

And thanks for offering to call, I appreciate it!