topic Re: Shared disk on Nimble for ESXi in HPE Nimble Storage Solution Specialists

Shared disk on Nimble for ESXi

General_Fault — Mon, 26 Oct 2020 11:21:13 GMT

So I've set up a vSphere HA with two hosts, but it needs shared storage.
Done the VMWare integration, iSCSI connections work, port binding is set up, initiators work, I can create VMFS datastores for each host without any problem.

I created a folder with VVols management type (in Nimble OS) and it's visible in VSphere, but when I try to create the VVol for the Datacenter in vSphere, it fails with "A general system error occured: invalid fault"

Maybe my approach is wrong for the need for shared storage, does it have to be a VMFS with multiple initiator access or a VVol? Or am I missing some configuration?

Re: Shared disk on Nimble for ESXi

General_Fault — Mon, 26 Oct 2020 13:09:55 GMT

Now I've allowed multiple initiator access on the two volumes I created for VMFS datastores, they seem to be working as shared storages now, visible for both host.

Still no idea why the VVol failed to create though...

Re: Shared disk on Nimble for ESXi

mamatadesaiNim — Mon, 26 Oct 2020 17:58:49 GMT

Thank you for reporting. Can you retry with disable HA and enable HA on the cluster a second time, please?

Re: Shared disk on Nimble for ESXi

General_Fault — Wed, 28 Oct 2020 11:19:26 GMT

Disabled HA and tried two ways:
- creating a "vVol01" folder on Nimble, then in vSphere: Storage > New Datastore > vVol > select "vVol01" > select both host > Finish
- in vSphere: Nimble Actions > Create VVol Datastore > select Nimble group > named vvol, hosts selected > size set > io limits default > Create

Both method ends with "A general system error occurred: Invalid fault" in vSphere.

Re: Shared disk on Nimble for ESXi

Nick_Dyer — Wed, 28 Oct 2020 11:37:12 GMT

Sounds like there's a mis-configuration somewhere. Are you using jumbo frames somewhere within your environment (if iSCSI)? Are they configured correctly end-to-end?

Another issue could be to do with vSphere certificates.

Best to give Nimble Support a call or email, they'll happily help you resolve it. here's their phone numbers: https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=a00050207enw

Re: Shared disk on Nimble for ESXi

General_Fault — Wed, 28 Oct 2020 13:24:09 GMT

Yes, I followed the integration guide which recommends the jumbo frames. It is configured on the switches, on Nimble and on the ESXi vSwitch and vmk ports.

The iSCSI connections seem to be working as I could add VMFS volumes to the hosts and it is accessible from vSphere, every VM is stored on these volumes and running fine.

Re: Shared disk on Nimble for ESXi

Nick_Dyer — Wed, 28 Oct 2020 13:27:17 GMT

VMware vVols can sometimes be tricky to get started on the VMware side due to HA and certificate issues. Please do give Nimble Support a call who will be able to quickly triage this for you.

Re: Shared disk on Nimble for ESXi

mamatadesaiNim — Wed, 28 Oct 2020 21:16:07 GMT

I see. So the problem is not with HA or creation of .vSphere-HA folder, but the creation of the vVol datastore itself. Got it. Good to know that you have a VMFS datastore mounted from the same array on the same ESXi server, and that works. As Nick mentioned, Nimble support can help you. If you want to try on your own, here are a few things to try:

check that vCenter and ESXi are using NTP for their time source, as vVols depend on certificates
check the output of these CLI commands on ESXi: esxcli storage vvol vasaprovider list; esxcli storage vvol protocolendpoint list; esxcli storage vvol storagecontainer list
check if there are paths to the PE device: esxcfg-mpath -L -d eui.<serial-no-of-PE-device>
check if there are errors in /scratch/log/vvold.log

If you are using vSphere 7.0, here's a blog post for setup/verification of vVols: https://community.hpe.com/t5/around-the-storage-block/vsphere-7-0-and-hpe-nimble-storage/ba-p/7087675

Re: Shared disk on Nimble for ESXi

General_Fault — Thu, 29 Oct 2020 14:41:48 GMT

Thank you both, I contacted HP support and they said the current settings look fine and it should work. They think it's a vmWare issue, but the case is not closed yet.

They tried the "openssl s_client -connect <Nimble IP>:8443" command, but it hangs on both host after only showing:
CONNECTED(00000003)

I've set up the NTP servers on the hosts and on vCenter. Also tried to refresh and renew certificates.

The "esxcli storage vvol vasaprovider list" command hangs for about five minutes, but returns:
<Group Name>
VP Name: <Group Name>
URL: https://<Nimble IP>:8443/vasa/version.xml
Status:
Arrays:
Array Id: NimbleStorage:*****
Is Active: true
Priority: 0

The "esxcli storage vvol protocolendpoint list" hangs for minutes and returns empty.
The "esxcli storage vvol storagecontainer list" hangs for minutes, but returns:
Datastore
StorageContainer Name: Datastore
UUID: vvol:0000000600004004-*****
Array: Unavailable
Size(MB): 0
Free(MB): 0
Accessible: false
Default Policy:

The vvold.log shows these popping up:
VasaSession::Initialize url is empty
VasaSession::DoSetContext: Empty VP URL for VP (***)!
Initialize: Failed to establish connection https://*.*.*.*:8443/vasa/version.xml
Initialize: Unable to init session to VP *** state: 0
VasaSession::GetEndPoint: with url https://(*.*.*.*8443/vasa/version.xml
SSL Async Handshake Timeout : Read timeout after approximately 25000ms. Closing stream <SSL(<io_obj p:0x000000febd6b9108, h:12, <TCP '*.*.*.* : 59026'>, <TCP '*.*.*.* : 8443'>>)>
VasaSession::GetEndPoint: failed to get endpoint, err=SSL Exception: SSL Exception: error:140000DB:SSL routines:SSL routines:short read: The connection was closed by the remote end during handshake., using default
VasaSession::GetEndPoint: failed to get endpoint, err=Connection reset by peer: The connection is terminated by the remote end with a reset packet. Usually, this is a sign of a network problem, timeout, or service overload., using default

Re: Shared disk on Nimble for ESXi

mamatadesaiNim — Thu, 29 Oct 2020 15:48:32 GMT

> They tried the "openssl s_client -connect <Nimble IP>:8443" command, but it hangs on both host after only showing:
CONNECTED(00000003)

This indicates there is a firewall between the ESXi/vCenter/Nimble for port 8443. It is essential that this port is kept open. Are you able to get firewall changes done, where this will work?

--m

Re: Shared disk on Nimble for ESXi

General_Fault — Thu, 29 Oct 2020 16:28:35 GMT

Are you sure? The vvold.log shows "Firewall: allowed access to port 8443"
And at the esxi firewall there's a "dynamicruleset" rule that allows outgoing TCP 80, 8443.
In vSphere the storage provider visible and online, the certification is there. On the hosts the vSphere certification is there.

I just tried disabling and enabling VASA provider on Nimble, refreshed the certification on the hosts, but still no luck.