new ESXi Host no connection to Nimble, no iSCSI path, device and PE

[Traq]

Hello,

i have a running Nimble with vmware integration (VASA) with using vvols.

Now i add a new ESXi Host and get no connection to Nimble. iSCSI configuration is 1:1 like the other Hosts and NCM ist installed. After adding the Nimble Discovery IP at "dynamic discovery" i see the targest at "static discovery" but no path, devices and protocol endpoints. I've already manually created the host initiator in the nimble gui "data access" but also without success and without the tab PE. The initiators of the other host have been created automatically. What is the right way?

thanks

[Nick_Dyer]

Hi,

Best thing to do is to call Nimble Support. They're awesome and will help you straight away.

[mamatadesaiNim]

Please do not create initiator groups manually for vVols!  This is a big no-no.  As Nick mentioned, you can work with Nimble Support.  Few high level steps for everyone trying vVols:

1. Double-check the vCenter, all ESX servers and the Nimble array are using NTP for time synchronization.
2. Register the Nimble vasa provider.  You can use the vCenter GUI or the Nimble GUI to do this.
3. Create a folder on the Nimble GUI, this will be the storage container that backs the vVol datastore.
4. From vCenter, then create the vVol datastore.  The dialogs will guide you to select the created Nimble folder as the backing storage container.

Depending on your Nimble OS version, we highly recommend you use our vcPlugin to perform these operations.  Refer to this post for a quick how-to: https://community.hpe.com/t5/HPE-Storage-Tech-Insiders/VVols-meet-Nimble-H5-plugin/ba-p/7045172#.XXp16ihKj-g

--m

[hpusedtomakegoo]

Having the exact same problem. New cluster, one host can see and work with nimble fine but the others cannot. Do you remember what your problem ended up being?

I cant call nimble support because i am between phones right now till i get a replacement.

if i try and add teh other hosts under "Data access" it says that the initator group is not managed by nimble and wont let me add any entries. But on the vmware side, i dont know how to authorize the hosts. Same thing, they appear connected on the iscsi level with a dynamic target, but i have no protocol endpoints on 2 of the three hosts. I had configured them all the same, but i guess i missed something.

[mamatadesaiNim]

Hello,

When you add a new host, please make sure it has vmknics that can ping all data IPs of the Nimble. Also, DO NOT create initiator group by hand.  Simplly "mount" the vVol datastore to the new host.  This is automatically create required initiator groups.

hth,

Mamata

[hpusedtomakegoo]

they can reach it and indeed i am able to mount and see a non vvol datastore. We did that some time ago to test connectivity. tried large packet pings as well. and all are fine and similarly connected.

:~] esxcli storage core device list
eui.b13b7469763e49ac6c9ce900e6864ddf
   Display Name: Nimble iSCSI Disk (eui.b13b7469763e49ac6c9ce900e6864ddf)
   Has Settable Display Name: true
   Size: 10240
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/eui.b13b7469763e49ac6c9ce900e6864ddf
   Vendor: Nimble
   Model: Server
   Revision: 1.0
   SCSI Level: 5
   Is Pseudo: false
   Status: on
   Is RDM Capable: true
   Is Local: false
   Is Removable: false
   Is SSD: true
   Is VVOL PE: false
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: supported
   Other UIDs: vml.0100000000623133623734363937363365343961633663396365393030653                                                                                  6383634646466536572766572
   Is Shared Clusterwide: true
   Is SAS: false
   Is USB: false
   Is Boot Device: false
   Device Max Queue Depth: 128
   No of outstanding IOs with competing worlds: 32
   Drive Type: unknown
   RAID Level: unknown
   Number of Physical Drives: unknown
   Protection Enabled: false
   PI Activated: false
   PI Type: 0
   PI Protection Mask: NO PROTECTION
   Supported Guard Types: NO GUARD SUPPORT
   DIX Enabled: false
   DIX Guard Type: NO GUARD SUPPORT
   Emulated DIX/DIF Enabled: false

 i did try removing and re adding the vvol datastore, but i get a warning that it cannot write to it. And the machines are NOT in the vvol initator group. 

i guess ill call nimble when i get access to a phone again and see what they say.

[mamatadesaiNim]

Did you remove the initiator group you manually created and any ACLs to PE that you may have added yourself?

Also outputs of these CLI can be helpful:

1. esxcli storage vvol vasaprovider list
2. esxcli storage vvol protocolendpoint list
3. esxcli storage vvol storagecontainer list
4. If you have installed NCM, /etc/nimble/nimble-mode.sh list

[hpusedtomakegoo]

thanks for the help here is the output.

NON working host:

[root@VMESX12:~] esxcli storage vvol vasaprovider list
Storage3-Group
   VP Name: Storage3-Group
   URL: https://10.0.4.50:8443/vasa/version.xml
   Status: syncError
   Arrays:
         Array Id: NimbleStorage:4445b3319b0835d7
         Is Active: true
         Priority: 0
[root@VMESX12:~] esxcli storage vvol protocolendpoint list
[root@VMESX12:~] esxcli storage vvol storagecontainer list
Storage3-VVolProduction
   StorageContainer Name: Storage3-VVolProduction
   UUID: vvol:0000000200004004-8445b3319b0835d7
   Array: NimbleStorage:4445b3319b0835d7
   Size(MB): 0
   Free(MB): 0
   Accessible: false
   Default Policy:
[root@VMESX12:~] /etc/nimble/nimble-mode.sh list

1-device=eui.b13b7469763e49ac6c9ce900e6864ddf isVvolPE=false num_array=1 mode=0 conn_num=2 iqn=iqn.2007-11.com.nimblestorage:esxiconnectiondatastore-v4445b3319b0835d7.0000145e.df4d86e6
    path_name=vmhba64:C0:T1:L0 conn_isid=00023d000001 conn_addr=10.0.4.50 remote_addr=10.0.4.52 local_addr=10.0.4.104 conn_mode=0 conn_state=logged_in
    path_name=vmhba64:C1:T1:L0 conn_isid=00023d000002 conn_addr=10.0.4.50 remote_addr=10.0.4.52 local_addr=10.0.4.105 conn_mode=0 conn_state=logged_in

Working host:

[root@VMESX13:~] esxcli storage vvol vasaprovider list
Storage3-Group
   VP Name: Storage3-Group
   URL: https://10.0.4.50:8443/vasa/version.xml
   Status: online
   Arrays:
         Array Id: NimbleStorage:4445b3319b0835d7
         Is Active: true
         Priority: 0
[root@VMESX13:~] esxcli storage vvol protocolendpoint list

eui.cdf29b0003e83ead6c9ce900e6864ddf
   Host Id: eui.cdf29b0003e83ead6c9ce900e6864ddf
   Array Id: NimbleStorage:4445b3319b0835d7
   Type: SCSI
   Accessible: true
   Configured: true
   Lun Id: eui.cdf29b0003e83ead6c9ce900e6864ddf
   Remote Host:
   Remote Share:
   NFS4x Transport IPs:
   Server Scope:
   Server Major:
   Auth:
   User:
   Storage Containers: 00000002-0000-4004-8445-b3319b0835d7
[root@VMESX13:~]
[root@VMESX13:~] esxcli storage vvol storagecontainer list
Storage3-VVolProduction
   StorageContainer Name: Storage3-VVolProduction
   UUID: vvol:0000000200004004-8445b3319b0835d7
   Array: NimbleStorage:4445b3319b0835d7
   Size(MB): 29360128
   Free(MB): 9776487
   Accessible: true
   Default Policy:
[root@VMESX13:~] /etc/nimble/nimble-mode.sh list

1-device=eui.b13b7469763e49ac6c9ce900e6864ddf isVvolPE=false num_array=1 mode=0 conn_num=2 iqn=iqn.2007-11.com.nimblestorage:esxiconnectiondatastore-v4445b3319b0835d7.0000145e.df4d86e6
    path_name=vmhba64:C0:T2:L0 conn_isid=00023d000002 conn_addr=10.0.4.50 remote_addr=10.0.4.52 local_addr=10.0.4.107 conn_mode=0 conn_state=logged_in
    path_name=vmhba64:C1:T2:L0 conn_isid=00023d000001 conn_addr=10.0.4.50 remote_addr=10.0.4.52 local_addr=10.0.4.106 conn_mode=0 conn_state=logged_in

2-device=eui.cdf29b0003e83ead6c9ce900e6864ddf isVvolPE=true mode=1 num_active_array=1 num_arrays_in_group=1 conn_num=2 iqn=iqn.2007-11.com.nimblestorage:storage3-group-g4445b3319b0835d7
    path_name=vmhba64:C1:T0:L0 conn_isid=00023d000001 conn_addr=10.0.4.52 remote_addr=10.0.4.52 local_addr=10.0.4.106 conn_mode=1 conn_state=logged_in
    path_name=vmhba64:C0:T0:L0 conn_isid=00023d000002 conn_addr=10.0.4.52 remote_addr=10.0.4.52 local_addr=10.0.4.107 conn_mode=1 conn_state=logged_in

as i said i was not able to manually edit the connector, so nothing to delete from nimble side under data access.

[mamatadesaiNim]

the vasa provider status on VMESX12 (non-working host) is syncError.  This needs to be fixed.  It can be from a number of problems.  What is the build and version of your ESXi?  (vmware -vl)

[imih]

We have the same problem. 

vmware vl: 

VMware ESXi 7.0.3 build-18644231
VMware ESXi 7.0 Update 3

Nimble OS version: 

5.2.1.800-930936-opt

[mamatadesaiNim]

@imih Are you able to contact Nimble Support?

The "same problem" can be a result of a number of different underlying issues, and Nimble Support can help work through and figure out what's wrong.

[BartRajchel]

We started to experience this issue after upgrading one of our hosts to 7.0.3.  VM removed it from their patch repository. We are currently reverting our hosts to 7.0.2.

I hope this helps.

[imih]

In our case we found a solution with the following parameters:

Nimble HF20H with software version 5.2.1.800-930936-opt.

VCSA version: 7.0.3, build: 18700403 

ESXi version: VMware ESXi, 7.0.3, 18644231

First, we updated all three of our Nimble HF20Hs. Then from the VMware integration section of Nimble web UI we had to remove and register the VASA Provider and the Web Client again because in VCenter page; Configure, Storage Providers settings displayed the previous version of Nimble plugin (5.0.10) and they were offline. Nevertheless, the continous iSCSI connections was working as it should be expected.

I hope this helps.

Istvan