Read IOPS on Nimble Storage

[csullivan43]

Hi An.Tran,

I did find some internal information indicating we did use that particular SSD drive at one point, however, we do not publicly provide these kinds of details. Provided that both you and your customer have signed an non-disclosure agreement, I am sure the local Nimble Storage SE's could provide more details.

However, the exact model of SSD is not relevant because of your assertion that a single SSD can only do 5000 4k IOPS is not true. By researching just the Intel models at the same link I provided you can find that a single SSD, not just that model can do significantly more than that. In effect, your primary question that there must be some secret method in CASL to achieve this performance is not based on accurate facts. If you don't accept that a single SSD can produce more than 5000 4k random IOPS then I will never be able to show you how we do it.

The figure you show is actually for write IOPS, which is confusing to me since this is a discussion about read throughput. Perhaps you can provide a figure that relates to read IOPS.

The reason that the addition of AFS to a CS-array does not increase the number of IOPS of the array is because we scale performance and capacity independently. The addition of the AFS, as Nick mentions, only serves to add capacity to the flash cache area which allows us to save a great deal more data on flash, meaning more read operations are served from that media. Performance is scaled by adding more and faster CPU's which allow us to then service more of those requests. If you look at the CS-series arrays, the primary difference in performance is the number and speed of the CPU's and not the amount of disks they have. Disk based storage systems scale by adding disks for random read workloads.

If you look at most traditional vendors controllers, these systems use CPU much differently. Also the code on these controllers was written many years ago and does not have the same ability as Nimble to take advantage of the larger number of cores in each modern CPU. They cannot provide the same level of parallel processing as in the Nimble operating system without completely re-writing their own operating systems. As a result, Nimble is more efficient and better able to process data quickly and efficiently. Another area where traditional disk based storage systems are not optimized for flash is in the way that they use the SSD in their storage systems. Because they are simply replacing HDD's with SSD's, they use the same RAID architecture on the SSD's. This means that they have to do many more operations to write data to the SSD which reduces the overall performance of the system and limits their throughput as Nick mentions also.

Because Nimble does not implement RAID on the SSD's in the system and uses them for reads instead of writes, we are able to provide much better throughput from SSD's than traditional architectures.

I am more than happy to continue this discussion or help you find someone locally who can explain this to you. However, as mentioned before, you need to agree that a single SSD can provide more than 5000 4k random IOPS. Unless we can agree on that point I'm afraid I won't be able to describe sufficiently how CASL and Nimble provide the documented performance we do.

Craig

[an_tran]

Hi Craig,

First, I'm very trust in single SSD can reach more than 5000 IOPS random read , I said one more time that I'm so believe it, OK?

But at here, the important thing is not about how many Random Read IOPS per single MLC SSD can provide, because of the traditional architecture, they have also based on the Random Read IOPS per SSD and absolutely it can reach up to 1.000.000 Random Read IOPS with 40K-50K Random IOPS per SSD drive (EMC, NetApp) even they just used the oldest O, OK?

I want to mention for 5000 IOPS per SSD drive in CASL because I really wanna underline about the POWER of CASL controller to leverage the very worst SSD drive with only the smallest number IOPS for reaching the very huge number of Random Read IOPS (even if the actually Random IOPS per SSD drive can more than 5000 IOPS so much, OK?)

I need what CASL did in controller to convince the customer. We can not said that "Sorry, we just proved the difference for Write IO by sequentializing the Random Write and for Read IO, this is our secret information.

Second, please don't tell anymore the benefit of CASL for scaling performance and capacity independent because I also understood it!! I My purpose when mention about upgrade AFS is I need to know about the difference for Read Random IOPS which almost based on the POWER of controller, not the number of SSD drive like the traditional architecture!!! OK bro

It seems you still not understood what I ask???

Finally, What I need is understand about what CASL did internally actually in Nimble Controller to leverage Random Read IOPS with any types of SSD drives even if the SSD drive is only 5000 or only 10.000, or 14.000 or 20.000!! OK bro???

If you said that because SSD can reach more than 40K IOPS so that it is quite easy to see how a CS-array can achieve very high random read throughput with only 4 SSD, it's very the same way other vendors did like I mention above NetApp, EMC can reach up to 1.000.000 Random Read IOPS with 40K-50K Random IOPS per SSD drive and IT'S NOT SHOW THE SPECIAL THING CASL CAN DO !!!!!

Thanks,

[Valdereth]

Wanted to chime, because this is something that I've often struggled to understand and in turn explain when speaking with people about Nimble magic.  I could be off base but I'll throw this out there.  When CASL writes the compressed, sequential 4.5MB stripes down to disk, that is taking a large number of input/output operations and encapsulates them within each stripe.  When a read request comes in, Nimble validates checksum, uncompress, and serves up the data. 

So with that decompression; does that also accelerate/inflate the IOPs provided to the requesting Host?

Anyone else want to weigh in on this? 

[rmalmberg-glc30]

Just to add on:

• Increasing the cache size does not necessarily increase performance. It certainly can increase application performance as more data can be in the cache, but from a performance ceiling perspective, it has no impact until the active data size exceeds the cache size.
• Nimble has Redirect-on-Write snapshots and no LUNs to manage; this results in the need for the metadata journal on the array to know where blocks of data are located and what blocks of data belong to which volume or snapshot.
  • This is the primary bottleneck under normal working conditions and the reason that performance is limited by the CPUs of the storage controller. Faster processors and more cores allow the storage controllers to find more data faster and decompress it to send the response to the requesting device.

[csullivan43 · Accepted solution]

An.Tran,

I think we are on the same page now. However, we also need to understand that we don't use lower performance SSD's because they don't have the throughput customers require and with slower SSD's we would need more of them to meet the performance requirements which would make the solution more expensive. We developed the CS-array and CASL to more effectively use the high performance of SSD's in a very efficient way. We use SSD to do what they are very good at, very fast read operations. Yes companies like NetApp and EMC can provide high throughput with their systems because the SSD's themselves are fast. However, at some point they are limited because of the way they arrange the data on the SSD, put RAID on the SSD, etc. Those numbers with millions of IOPS are also with multiple controllers in scale-out configurations with many SSD's.

So the magic of CASL is this:

• Use SSD's for caching data for reads
• Use intelligent caching algorithms, aggressive caching, and volume pinning to increase the number of reads that actually come from SSD.
• Increase the size of the cache with AFS to allow for more data to be stored increasing the likelihood that a read request is served from SSD.
• Use the SSD to store metadata to make the lookup of data blocks faster.
• Take advantage of high speed multi-core processors to make the management of data, compression, IOPS faster.

I hope this helps.

Craig

[an_tran]

Hi Craig,

OK thank a lot for your explanation.

I would like to summarize everything:

Random Read IOPS on CASL are leveraged by the both of Controller and the types of SSD drives.

But the most of SSD drives on Nimble Storage can deliver the Random IOPS more than 30.000 IOPS so that Random Read IOPS will absolutely depend on the Difference on Processors each CS-array to calculate exactly number for Read Random IOPS

Thank you again