Array Performance and Data Protection
cancel
Showing results for 
Search instead for 
Did you mean: 

Nimble Storage Deduplication?

 
SOLVED
Go to solution
Highlighted
hoangdv105
Occasional Advisor

Nimble Storage Deduplication?

Hi all,

I want to understand how deduplication of nimble storage work  but i can't see any document about that. Maybe i miss to find that. Please someone can tell me about that !

Thanks !!

11 REPLIES 11
Highlighted
rugby0134
Esteemed Contributor
Solution

Re: How nimble storage deduplication ?

Currently the Nimble OS only support LZ4 compression. Because the array has such a low cost per GB, we don't need to deploy De-duplication to save storage space.  Deduplication for primary storage only helps save storage in a couple specific workloads.  Deploying deduplication for VDI clones workloads for example don't save you much.

Highlighted
hoangdv105
Occasional Advisor

Re: How nimble storage deduplication ?

Thank you, but can you explain for me how LZ4 compression work ?, and how much percent capacity space reduced when compressed ?

Highlighted
rugby0134
Esteemed Contributor

Re: How nimble storage deduplication ?

RealTime Data Compression: LZ4 explained here's a good explanation.

Highlighted
rlabarre65
Occasional Advisor

Re: How nimble storage deduplication ?

Hoang DV.  Given the fact that we have Infosight, and thus an insane amount of detail about a customers data we can tell you that we average around 50% compression (as a total customer average).  However, as you know all workloads are different and your individual mileage may vary.  While databases tend to skew the graph higher, video files etc move it in the opposite direction (as they offer no compression). We can see file share data being highly compressable at times as well.  Exchange data, OS Volumes etc are compressable as well, to what extent will again, vary based on your particular data sampling.

Hope that helps.

Highlighted
tmcmullin51
Advisor

Re: Nimble Storage Deduplication?

Hoang,

If you really want to know the gory details, here are more some good articles:

LZ4 (compression algorithm) - Wikipedia, the free encyclopedia

lz4 - Extremely Fast Compression algorithm - Google Project Hosting

Wow, LZ4 is fast! - Adrien Grand

My point in sharing this is that while there are other algorithms that can provide slightly better compression ratios, Nimble chose LZ4 because of its speed - which is critical when you're designing an enterprise disk array.

Is there a particular type of workload in which you're interested?  I could probably look up some average compression ratios for you.

hoangdv105
Occasional Advisor

Re: Nimble Storage Deduplication?

Thanks about your support, this really helpful for me !

Highlighted
jwagner137
Advisor

Re: Nimble Storage Deduplication?

I know this has already been flagged as answered but aren't we talking two entirely different things here?

Compression and deduplication aren't the same thing at least not from my understanding of what each does and how it does it.

Highlighted

Re: Nimble Storage Deduplication?

Hi John,

Thank you for bumping this thread. We actually have a great blog post discussing Nimble's entire data reduction approach following the launch of our all flash arrays. It goes into deduplication, inline compression, zero-pattern elimination, and zero-copy clones. There is also an in-depth technical blog post for our Nimble OS 3.0 blog that discusses dedupe from an engineering perspective.

I hope this provides a better picture of what you're asking about.

All Flash: Data Reduction Beyond Dedupe | Nimble Storage

Dedupe – An Engineer Looks Under the Hood

Cheers,

Kat Usavage - NimbleConnect Admin

Highlighted
jwagner137
Advisor

Re: Nimble Storage Deduplication?

Kat,

Thanks very much for this info. Nice to know that this is coming!