topic Re: Is deduplication coming to Adaptive Flash? in Array Performance and Data Protection

Is deduplication coming to Adaptive Flash?

gary_martin — Wed, 24 Feb 2016 10:39:34 GMT

With the release of Nimble All Flash, including deduplication and the statement regarding Nimble OS being the same across both Adaptive and All Flash arrays. Does this mean deduplication is coming to Adaptive Flash?

It would probably mean shifting data around in order to get it on a pass through inline, but it really would fill that small hole when comparing Nimble with other solutions. Don't get me wrong here, compression and thin provisioning have given us significant savings in our environment, but it's always nice to get 'more' efficiency.

References:

http://www.nimblestorage.com/blog/nimble-flash-arrays-leapfrog-pure-storage-emc/

"Data Reduction – Nimble has implemented inline variable-block deduplication, compression, zero-block elimination, thin provisioning and zero-block cloning that achieve similar data reduction to Pure and higher data reduction compared to XtremIO."

"And because Nimble All Flash arrays and Adaptive Flash arrays run the same version of NimbleOS, we can now provide flash for every enterprise application in the data center – without compromise."

Gary

Re: Is deduplication coming to Adaptive Flash?

Nick_Dyer — Wed, 24 Feb 2016 12:15:17 GMT

Hi Gary,

We cannot post definitive, guaranteed, forward-looking statements on a public forum... however having said that, it is planned to roll out that feature on Adaptive Flash in a future release.

Bear in mind that dedupe is only useful for a certain classification of data (such as VMs or file images); yet it can have an overhead on CPU/memory performance.

Re: Is deduplication coming to Adaptive Flash?

gary_martin — Wed, 24 Feb 2016 13:15:04 GMT

Thanks Nick, probably might happen is a nice response. Much better than "we can't tell you anything until we can tell you everything".

Nimble seem like a company that wants simplicity in message, so currently it's controller X is capable of Y IOPS. When other factors can influence those numbers it muddies the message a bit (i.e. controller X is capable of Y IOPS with feature Z enabled). I am sure you guys will find a neat way to represent this.

With regard to All Flash deduplication, is that per volume or per array or per array group? Is it a system wide setting or more granular?

Thanks again

Gary

Re: Is deduplication coming to Adaptive Flash?

Nick_Dyer — Wed, 24 Feb 2016 15:28:13 GMT

You're absolutely right sir, and is one of the main reasons we didn't introduce dedupe on the Adaptive Flash platforms to start with; there's little value promising good performance and latency, unless you have a write intensive workload which potentially cripples performance and spikes latency at >30ms whilst it runs. Also, dedupe on a platform which already returns an excellent $/GB thanks to RAID3P and high capacity NL-SAS drives is a tad irrelevant (in my opinion).

The way that dedupe is deployed is down to a couple of new constructs we're introducing into NimbleOS; Folders and Application Profiles.

Application Profiles are essentially an extension of Performance Profiles. It is a way to turn on dedupe on the application layer, to ensure that dedupe is not being run on apps that frankly will yield poor space savings and burn CPU cycles.

Folders are a logical grouping of volumes together, in which an Application Profile is deployed on top to turn on/off dedupe/compression/encryption/snaps/replication etc. It's also an easier way to manage 1000's of volumes within the Nimble UI. Folders and Application Profiles will also be the constructs powering our production VVOL support.

Hope that helps for now. When we near the GA release of NimbleOS 3.0 we will have more detailed tech blogs on how all of the above works!

Re: Is deduplication coming to Adaptive Flash?

mkieran59 — Wed, 24 Feb 2016 18:40:24 GMT

We'll have a lot more detail on the new dedupe capabilities in a couple of blog posts tomorrow (Thursday), both here on NimbleConnect and on the Nimble corporate blog.