Array Performance and Data Protection
Showing results for 
Search instead for 
Did you mean: 

Performance on Large-Quantity, Small-Size Files

Go to solution
Occasional Contributor

Performance on Large-Quantity, Small-Size Files

Hi to all,

I am just wondering how Nimble would perform on small-size, but numerous files.

Let's say, I'll be using Nimble for file servers.

Clients connecting to the server create and modify files of small sizes (less than 100KB), but there are millions of it.

What can be the best performance policy to apply?

Or what approach can be done to achieve optimized performance?



Re: Performance on Large-Quantity, Small-Size Files

I think ultimately the success or failure of this requirement would be down to the decision on what to use for host-side SMB/NFS shares - ie Linux shares or something like Windows 2012 R2. The latter is pretty darn good these days as file servers, which I never thought i'd say!

The ultimate answer here is to test it out - i'm sure your local Nimble SE would only be happy to help!

Occasional Contributor

Re: Performance on Large-Quantity, Small-Size Files

Thanks Nick.

Actually, I was testing Nimble and another storage (3-letter word..) for the comparison.

Say I run a script that creates certain thousands files, 1KB size each for each file.

Nimble took 5 mins to complete while the other was around 3 mins only.

I tried to configure Nimble with different settings but it wouldn't give an equal or better result.

The other storage was old so I was expecting that I would be left behind on this testing.

By the way, both storage were connected on ESX host with Windows 2008 R2 guest OS VM.

So I am really wondering where the "wrong" is.


Re: Performance on Large-Quantity, Small-Size Files

My initial hunch is the script is creating a lot of sequential IO to the array. Was it a CS200 or CS400 system? What did the array UI's performance tab look like at the point of the script being executed?

Esteemed Contributor

Re: Performance on Large-Quantity, Small-Size Files

Lots of you have NCM setup, load balancing selected correctly, multiple 1-GB NIC's or 10GB setup?  Windows AV turned off.  I have seen and made several SMB shares that have done 400-500 MB/s, so i would expect your hitting a bottleneck in the iSCSI network

Occasional Contributor

Re: Performance on Large-Quantity, Small-Size Files

It's CS220, version The throughput performance was hitting average of 1.4MB/s random write and 0.04MB/s seq write. IOPS performance was on 285 random write and 0.26 seq write. I only gave the write ones because Nimble seems very good on read side.

For the setup, I believe NCM is available on Nimble Ver.2, no NCM. Connections for Nimble-to-ESX is 10Gb. By the way, I forgot to mention that we have test clients that we use to simulate similar environment on our company. Currently, we don't have AV installed yet.

Clients<--100Mb Hub--1Gb Switch-->VM server(Windows File Share)<--10Gb-->Nimble Storage.

This structure is same with the other storage brand setup, except that it connects with 4Gb FC protocol.

We also performed benchmarking on the server side where Nimble volume is directly presented.

Results showed that Nimble is way better on large block sizes files, but not on small one (4K size).

I tried using Nimble volumes as ESX datastore, iSCSI target volume, and RDM.

On those approaches, I applied different performance policies just to figure out which combinations will give results comparable to the other storage.

I have been seeing a lot of impressive reviews about Nimble, and believing it.

However, it is my bosses who I am wanting to make believe that Nimble is better than the current storage we have.

This is why I need to figure out where the problem is.

(Quite long, have patience reading this.)


Re: Performance on Large-Quantity, Small-Size Files

Good morning,

Thanks for the detailed environmental analysis. If you have VMware Enterprise licensing or above its worthwhile installing our Nimble Connection Manager for VMware PSP which will enhance multipathing & path management. It can be found on Infosight. Also if this is a fresh, new install it's worthwhile looking at NOS 2.1 as it provides enhancements over 1.4.11.

Are you presenting the storage volumes to the file share via in-guest iSCSI, or via VMware's VMFS?

Do you have a support case open with Nimble to track this issue? If so, please post the case number. If not, please do so!

Another test would be to load IOMeter and emulate the same test in that tool to see if you can recreate the problem, which may give us some more clues as to where to look next.

Trusted Contributor

Re: Performance on Large-Quantity, Small-Size Files

In regards to the Vmware configuration... did you have multiple NIC's in the host, one to one vmkernel port to vmnic relationship, and if not using the NCM, did you change MPIO from the default?

Re: Performance on Large-Quantity, Small-Size Files

If what I am reading is correct, you don't have NCM - which is a critical piece of the puzzle when it comes to Nimble performance with VMware. You will see an order of magnitude difference using it vs. not using it. Even RR does not come close to the NIMBLE_PSP_ROUTED path selection policy.

Virtualization Lead Architect
BEMA Information Technologies