Performance Impact of a Failed disk in a RAID 10 Array

[HDMCS]

Hi All,

We have a failed disk that is being replaced under warranty. In the meantime is there anything we can do about the performace degradation?

The failed drive is one of six  900 GB SAS HDDs in a RAID 10 volume connected to a HPE Smart Array P408i-a SR Gen10.

The write performance is so degraded as to make it unusabled (its the storage used for virtual disks for a standalone hyperv server)

I've been using the ATTO DIsk Benchmarking tool to look at this.

For IO sizes between 512 B and 256 KB Read Performance is between 34 MB/s and 5 GB/s

This is comparible to a volume on another  server with a similar RAID configuration

Above that size,  read performance drops to around 700 MB /s

Write performance maxes out at 680 MB/s

My understanding is that RAID 10 is a mirror of two stripe sets. With one of the stripe sets unusable with a failed disk I might have hoped to get the performance of a single stripe set? Instead I'm getting next to nothing. Is there a way of telling the controller to do ignore the other stripe set?

Matt Hardy

[Ihaqueit]

RAID 10 can stand a single drive failure per array

RAID 10 only reads the surviving mirror and stores the copy to the new drive you replaced. Your usual read and write operations are virtually unchanged from normal operations.

Better to replace as soon as possible.

[HDMCS]

Thanks for your response. The replacement drive should be with us by the end of the day so it will soon be a moot question but are you saying that the extreme performance degradation that we're seeing is NOT to be expected?

Matt Hardy

[Torsten.]

The drive is probably not really dead and is slowing down the system.

Remove the disk now and insert the new disk once it arrives.

[ksram · Accepted solution]

Hi,

The Performance slowing down due to Drive is less expected.

However you can try to update the Controller + Drive Firmware to the latest available, which may have critcal fixes and or enhancements available.

And yes, you may try removing the failed Drive to check if that helps.

Make sure to take regular Backups.

Thank you
RamKS

[HDMCS]

Many thanks for your suggestions. The drive is now replaced and rebuild is completed.

I'm able to run virtual machines again however the performance benchmark still falls far short of that on a similar server's array. As I don't have a benchmark from before the incident it may be that this is the unaffected performance and that the issues we saw on the VMs on Saturday after we'd rebooted for updates was due to the system attempting rebuilts. I will however follow the suggestion to update the Controller + Drive Firmware to the latest available to see if that helps

Matt Hardy