ESXi host unable to mount datastores

[Dan Booth]

I have 2 identical ESXi 6.0 hosts trying to access datastores on Nimble via iSCSI and while one works fine the other now cannot mount the datastores (even though when first setting up iSCSI a week ago I managed to mount one of the datastores, it has since dissappeared).  The Nimble shows initiators from both hosts connected to individual volumes, and examining the iSCSI Storage Adapter under ESXi shows the datastores under Devices, but they don't appear under Confguration > Storage.  If I try Add Storage in vSphere Client I can select the missing Datastores on Select Disk/LUN but then the next screen (Current Disk Layout) hangs and never populates the bottom half and the Next button stays greyed out.  If I try to create a new Datastore and attach to the problem host using the Nimble plugin under vCenter the task hangs at 69% then eventually fails with the error 'Cannot change the host configuration'.  Can anyone advise how to fix this issue please, thanks?

[Giya]

Hi Dan Booth.

Couple of Questions:

What version of vSphere are you running?  Are you using the vSphere client(fat client) or Web Client?  Anything done to the host so far (reboot., etc.)?  Are both hosts mounting the same datastore(s) or different ones? When looking at the device under the iSCSI storage adapter-devices, what does the “operational state” column show for that device?

Check and verify the device id  shows on the problem host

1- Note the device id.   SSH to your problem host and run the following commands:

2- List all the disk devices on the host 
ls -lha /vmfs/devices/disks/

   2a-look for your unique device id
   Example of device id: “naa.12345678910111213141516171c1e202”

3-run partedutil on your unique device  identifier and examine the existing partition table on the device
partedUtil getptbl /vmfs/devices/disks/ <your disk id >

Example: partedUtil getptbl /vmfs/devices/disks/naa.12345678910111213141516171819202

Provide the output it returns

[markchapman]

Are you using jumbo frames? I had a similar issue where one host was misconfigured, it could see the volumes but not the datastores contained within!

[mamatadesaiNim]

So, at some point, both volumes were mounted on both hosts as 2 VMFS datastores, correct? Few things to check:

1. Are both volumes marked "multi-initiator access" as enabled?
2. After performing a rescan from both ESX hosts, do you see paths to both devices?  What does "esxcfg-mpath -L" look like?
3. What does /scratch/log/syslog.log contain right after the rescan, on each ESX host?
4. Do the volumes show up as VMFS snapshots by chance? Check the output from "esxcli storage vmfs snapshot list".  This is most likely the case.  Follow this VMware KB: https://kb.vmware.com/s/article/1011387

[ashraf-fouad]

it seems to be not consumed