Splunk / Logstash on Nimble Storage

[cfeskens93]

We are currently considering moving a splunk indexer onto our CS460.  I'm curious as to whether any other customers have deployed log indexing services such as splunk or logstash on Nimble, and what kind of recommendations you might have in terms of configuration, performance policies, etc.

[Nick_Dyer · Accepted solution]

Hello Casey,

Yes we absolutely do have a few customers who have chosen Nimble for their Splunk deployment (one being a pretty large Telco in the US no less). There's no Best Practice Guide for this deployment yet - but the first step is being certified for Splunk which happened a couple of weeks ago.

From what i've heard internally we've got some deployments with Indexers and Searchers on the Nimble platform, and some with just Searchers (Indexing being done on the host side).

Here's some notes taken from an internal discussion on the subject recently:

• Splunk IO patterns will vary depending on the use case but there are basically two types of IO. If there is a lot of search, then a lot of random read IO. If there is little search and just archive – then lots of larger sequential write IO for Index.  So the answer is..it depends but both will be present. It may be better to separate the Index and Search functions. They will be on separate servers.
• Search heads can be virtualized (VMware Performance policy) and Index servers will generally be Physical per Splunk best practices (so a custom policy with 32k block size) for the index volume(s). Aggressive Caching may be applicable to the Indexing servers so may need bigger SSDs or an All Flash Shelf.
• The real cool thing with Nimble is that we can transparently handle both forms of workloads within Splunk using CASLs intelligence, unlike an Flash Only (aka All Flash Array) which will not be good for the Index function. Adaptive Flash in action
• We will get some compression on top of Splunk’s (zlib) in the Index - somewhere in the region of 20-30%.

Hope this helps!