Read IOPS on Nimble Storage

[an_tran]

Hello,

We used to read about how Nimble can leverage the write performance by sequentializing random write data to SATA HDDs

At the result, each 7k2 HDD can reach up to 10.000 write IOPS (CS700) and the difference about write performance between CS-array is depend on the amount of CPU core for the speed of sequentializing random write

But for the Read IOPS performance on Nimble Arrays, it doesn't base on the speed of CPU like write IOPS performance.

And the Read Way is go to NVRAM, DRAM first and then SSD.

If MLC SSDs can be up about 5000 IOPS per drive, so that 4x SSDs will be 20.000 Read IOPS on each Array? Is it the same performance on all CS-array because they have the same 4x SSD drives?

Please explain it more detail for me.

Thanks

[Nick_Dyer]

Hi there, thanks for the question.

We have a great youtube video which describes the read/write IO path for random and sequential data which will most likely answer the majority of your questions. But to summarise, Nimble's CASL file system does not bottleneck on SSD (we get far more than 5000 IOPS per SSD), the bottleneck for read/write IO is compute within the controller - therefore it's feasible to achieve 15K random read IOPS on a CS2xx, yet with the same SSDs in a CS500 we can achieve >90K random read IOPS.

[an_tran]

Thanks for your answer.

But i still don't understand. I watched your video many times.

For the write IO, it's easy to know.

But for the read IO, what exactly are computed in the CS controller to reach >30K random read IOPS on CS300 or > 90K random read IOPS on CS500 with only 20.000 backend IOPS on 4 x SSD drives (4x 5000 IOPS which you mention about SSD performance) ?

I mean for 100% read, if we got 90.000 IOPS to controller, after that 90.000 IOPS will come to SSD drive with only 20.000 IOPS? Why don't have the bottleneck here? Should the performance be only 20.000 IOPS?

If not, could you please explain more detail.
Thanks so much

[peder72]

Hi Tran,

An SSD drive can deliver far more than 5000 read iops, more like 50K iops when using 4K blocks. SSD's are slower on writes because of write amplification and garbage collection, to put a raid on top also make the problem worse.

Nimble use SSD's for what they do best, -random reads without raid overhead. The highest number Ive seen in a CS700 using only 4x SSD's is 185K iops on random read, thats close to 45K iops per drive and its all because of CASL, our optimized filesystem where every type of media is optimized and utilized in the most efficient way.

Thanks

[an_tran]

Hi Peder,

I knew CASL is very good and optimized for every type of media and adapt to various block size, right?

But at here, I need to understand what exactly CASL did in controllers to make it difference on each CS model even all CS model (except CS700) used the same SSD drive type MLC SSD and same number of SSD drive (4 drive)

You said "An SSD drive can deliver far more than 5000 read iops, more like 50K iops when using 4K blocks" . I don't agree with you , I believe that the 5000 random read iops on each normal MLC SSD at 4K block size.

Thanks,

[znewman48]

Many consumer level SSDs, even at the budget level, can do 90K+ IOPS when doing 4k random read. 

[csullivan43]

Based on the information HERE, an MLC SSD drive can do 38000 random read IOPS.

With that information, it is quite easy to see how a CS-array can achieve very high random read throughput with only 4 SSD.

[an_tran]

Hi Sullivan

Could you ensure that SSD drive use on Nimble Storage, because there are many types of MLC SSD on the storage market.

You just guess about type of SSD that using on Nimble Storage and can you prove it or any public document told about the number IOPS per SSD?

For me, I searched so many Nimble Document and found out the MLC SSD can reach about ~5000 random read IOPS (Nick Dyer also confirmed this above and he told that the random read based on controller??)

For example: You can see this picture on Nimble Presentation. The IOPS per MLC SSD can reach 5000 IOPS, IOPS per 7.2K HDD on CS700 can reach 10.000 IOPS.

It seems you're not correct?

In addition, if Nimble Storage based on the number of SSD drives and types of SSD drive for random read IOPS, it won't make the difference between Nimble Storage and other vendors. Besides, if Nimble Storage based on number of SSD drives and types of SSD drive, it also increases the random read IOPS when we upgrade CS with AFS ?? It's very similar to the traditional architecture and not specially.

Finally, what I need is what actually Nimble Controller did for leverage random read IOPS without depend the number of SSD drives and types of SSD?

[Nick_Dyer]

Hello,

There is no technical limit of 5000 IOPS per SSD for MLC flash. As the other posters mention, it's entirely feasible to get 40-50K IOPS random read (4k blocks) from a single MLC SSD. It's more the case that other, legacy enterprise storage vendors cannot achieve that level of performance due to their traditional, disk-based file systems, which is why it's capped at 2500-5000 IOPS per SSD (e.g. EMC and Dell, respectively).

Also, adding more SSDs to a Nimble solution does NOT increase the IOPS, it merely increases the working set for hot data (ie we can service more blocks from SSD to ensure ~97% flash read, and to achieve the top-end IOPS figure the array is capable of).

It might be best to have one of our local SEs reach out to you to discuss this in more detail; it's very hard to describe over a forum. Can you please message me with your contact details and location?

[an_tran]

Hi Nick,

I'm also SE at Viet Nam. I need this information to explain for my customer.

I asked someone from Nimble at Singapore, but so far no one could explain more detail for me what detail Controller did to leverage the RANDOM READ IOPS.

I mean what the difference thing inside Nimble controller with other traditional vendor's controller.

I knew your information "Also, adding more SSDs to a Nimble solution does NOT increase the IOPS". I agreed that, therefore I need exactly why Nimble can increase RANDOM READ IOPS without adding more SSDs?

You mention about disk-base file systems so that could you give me some document or some link about that they're cause to limit the IOPS per single MLC SSD to 2500-5000 instead of 40K-50K?