NCM for VMware - single array purpose

[jayst136]

i'd like to clear up a frequent discussion i have with customers regarding deployment of the NCM for VMware and its benefits in single array setups.

I'm looking to get a more detailed listing of the actual features(or adjustments it makes) the NCM vib provides? All i can think of is the PSP policy for Nimble volumes, but i'm also curious what's being adjusted(if any) compared to default round-robin PSP.

[Nick_Dyer]

Hi

It's all in the VMware Integration Guide which is available on Infosight if you want details.

In short, the Nimble Storage Connection Manager installs two components:

‒Nimble Storage Connection Service (NCS) automatically calculates and maintains the optimal number of iSCSI sessions from host to the storage group balanced across the host's NICs for a HPE Nimble Storage device (very important to ensure enough paths and queues are available for optimimum IO).

‒Nimble Storage Path Selection Plugin (PSP) for VMware Pluggable Storage Architecture. Automatically directs the I/O request for a HPE Nimble Storage device to the most favorable route. This is enhanced from the poor VMware Round Robin policy which only switches paths every 1000 IOs by default.

The best practice is to install NCM for every server that's connecting to Nimble - be it VMware, Windows, Linux, Docker etc. It saves a lot of time and headaches. It's not strictly needed for Fibre Channel, but it should be a no-brainer for iSCSI environments.

[jayst136]

yes but i'm still considering the effort needed to install and maintain a ESXi version-specific  driver and its benefits, compared to not deploying it. It depends on the environment, but knowing more details could make the descision easier.

When i deploy a single iSCSI array i setup iSCSI connections to the array from within VMware. I get 2 sessions, one to each IP (even/odd loadbalancing configured on the array), PSP begin VMware's default RR. both paths are active. What would NCS be able to optimize here? I would get 2 sessions whether or not i'd install the NCM right?

I do get the advantage of the PSP that gets deployed, but only when you're looking to rule-out possible suboptimal configuration. Again, in short, NCM and it's PSP feature is to ensure Nimble volumes immediately (without further configuration) always get 1) the round-robin loadbalancing and 2.) the IO operation limit is adjusted for nimble volumes from 1000 to 1 for better performance. (https://kb.vmware.com/s/article/2069356)

With the NCM not being strictly needed for FC, i assume that's because of the concept of "sessions" does not apply? the PSP setting the IO operation limit from 1000 to 1 is still relevant in FC, which could be a very good reason to still deploy it, right?

[Nick_Dyer]

It's not quite as simple as that. If you don't use NCM, then you'll only ever have 2 paths (as you say) which is in most cases going to be sub-optimal. the Connection Service part of NCM will ensure that it will create more paths dynamically per volume on the fly, up to 8 connections in total. It does to ensure optimal IO and low queue depth with the volume is in need of it. This is very important especially for consistent performance and latency.

Standard VMware round robin isn't truely active/active, it's active/standby with 1000 IOs per path. Unless this is changed manually via the CLI (which isn't persistant across a host reboot) then you'll only ever have a single network path being used at any point in time with VMware alternating 1000 IOs each time. Our PSP removes that and ensures true loadbalancing across the paths. It also persists the change across reboots.

It also tunes various timeouts in VMware to Nimble's best practice configuration - something otherwise which will need to be done manually, and sometimes again doesn't persist when performing a host reboot.

The three examples above are the top three issues we see constantly in Nimble support - and NCM for VMware alliviates all three of them.

Encourage you to have a read through the VMware Integration Guide if you want more information. We'll be launching new VMware/Nimble integration in the near future and NCM will be a big part of that integration stack. The first part of that being synchronous replication which is due imminently.

Finally, VMware use ALUA for FC failover and path management - which is far more resillient than iSCSI - hence why it's not so much of a demand in FC environments. In fact, if it's a Windows based FC environment, installing the Windows Toolkit injects an MPIO driver but there is no 'NCM' functionality.

[jayst136]

thanks Nick. I certainly read some things i did not know and this will make it easier to proxy the story to customers!

One thing about the presistence of the PSP loadbalancing configuration, i think it used to be non-presistent but should be persistent after reboots in more recent versions. I had trouble confirming your statement about that. I'd be really suprised if this is still non-persistent in ESXi 6.0+ , i have several non-ncm (non-nimble) users using this setting actually across different storage vendors. But, i have not tested/verfied this with my own eyes :)

see for example: https://communities.vmware.com/thread/395841

[Thomas_Lam_HK]

I personnally had deployed hundred of ESX servers with NCM installed,  confirm RR policy setting to Nimble LUNs are persistent even after server reboot...

[medwards999999]

Hi Nick, I have a customer with a 6 month old Nimble system using iSCSI with the Nimble NCM on their ESXi 6.5U2 hosts, they are very concerned as the ESXi hosts are having occaisional and seemingly random path redundancy lost alerts and associated lun OFFLINE events in the vmkernel logs, is it possible that these may be being being generated by the NCM disconnecting paths after a period of high storage utilisation? We have calls logged (since Oct last year) with VMware, Dell (hosts) and Nimble but so far no one has been able to pinpoint the issue.

Any help or pointers would be very much appreciated.

[Nick_Dyer]

Can you provide me the Nimble support case ID via private message please... i'll check it out on our systems for you.