Array Performance and Data Protection
cancel
Showing results for 
Search instead for 
Did you mean: 

Nimble Storage Deduplication?

SOLVED
Go to solution
hoangdv105
Occasional Advisor

Nimble Storage Deduplication?

Hi all,

I want to understand how deduplication of nimble storage work  but i can't see any document about that. Maybe i miss to find that. Please someone can tell me about that !

Thanks !!

11 REPLIES
rugby0134
Esteemed Contributor
Solution

Re: How nimble storage deduplication ?

Currently the Nimble OS only support LZ4 compression. Because the array has such a low cost per GB, we don't need to deploy De-duplication to save storage space.  Deduplication for primary storage only helps save storage in a couple specific workloads.  Deploying deduplication for VDI clones workloads for example don't save you much.

hoangdv105
Occasional Advisor

Re: How nimble storage deduplication ?

Thank you, but can you explain for me how LZ4 compression work ?, and how much percent capacity space reduced when compressed ?

rugby0134
Esteemed Contributor

Re: How nimble storage deduplication ?

RealTime Data Compression: LZ4 explained here's a good explanation.

rlabarre65
Occasional Advisor

Re: How nimble storage deduplication ?

Hoang DV.  Given the fact that we have Infosight, and thus an insane amount of detail about a customers data we can tell you that we average around 50% compression (as a total customer average).  However, as you know all workloads are different and your individual mileage may vary.  While databases tend to skew the graph higher, video files etc move it in the opposite direction (as they offer no compression). We can see file share data being highly compressable at times as well.  Exchange data, OS Volumes etc are compressable as well, to what extent will again, vary based on your particular data sampling.

Hope that helps.

tmcmullin51
Advisor

Re: Nimble Storage Deduplication?

Hoang,

If you really want to know the gory details, here are more some good articles:

LZ4 (compression algorithm) - Wikipedia, the free encyclopedia

lz4 - Extremely Fast Compression algorithm - Google Project Hosting

Wow, LZ4 is fast! - Adrien Grand

My point in sharing this is that while there are other algorithms that can provide slightly better compression ratios, Nimble chose LZ4 because of its speed - which is critical when you're designing an enterprise disk array.

Is there a particular type of workload in which you're interested?  I could probably look up some average compression ratios for you.

hoangdv105
Occasional Advisor

Re: Nimble Storage Deduplication?

Thanks about your support, this really helpful for me !

jwagner137
Advisor

Re: Nimble Storage Deduplication?

I know this has already been flagged as answered but aren't we talking two entirely different things here?

Compression and deduplication aren't the same thing at least not from my understanding of what each does and how it does it.

Re: Nimble Storage Deduplication?

Hi John,

Thank you for bumping this thread. We actually have a great blog post discussing Nimble's entire data reduction approach following the launch of our all flash arrays. It goes into deduplication, inline compression, zero-pattern elimination, and zero-copy clones. There is also an in-depth technical blog post for our Nimble OS 3.0 blog that discusses dedupe from an engineering perspective.

I hope this provides a better picture of what you're asking about.

All Flash: Data Reduction Beyond Dedupe | Nimble Storage

Dedupe – An Engineer Looks Under the Hood

Cheers,

Kat Usavage - NimbleConnect Admin

jwagner137
Advisor

Re: Nimble Storage Deduplication?

Kat,

Thanks very much for this info. Nice to know that this is coming! 

alucorp11
Occasional Visitor

Re: Nimble Storage Deduplication?

Hi Hoang,

Deduplication, specially inline dedup is excellent, not just for saving storage space, but also performance. Less data to write, plus more data fitting in your cache, all adds up to better performance.

For VDI, I see 30:1 data reduction, though it is with a competitor's array and not Nimble, since Nimble does not have deduplication.

rugby0134
Esteemed Contributor

Re: Nimble Storage Deduplication?

Nimble does have DeDup in the 3.1 code release.    Buy an All-Flash Nimble array and you're golden!