topic vnic's for in guest iscsi in Array Setup and Networking

vnic's for in guest iscsi

lindy37 — Wed, 24 Feb 2016 18:39:05 GMT

I have VM's on VMware that I am setting up in-guest iSCSI. These guest iSCSI NIC's are riding the same real NIC's on the VMware Host that is used for iSCSI from VMware to the Nimble.

I give each VM two vnic's for iSCSI so I can use MPIO. My question is does Nimble have a recommendation for the vnic settings in the Windows VM?

I turned ON Jumbo frames since my physical network from the VMware host is using Jumbo frames. What about all of the TCP Off Load options in the vnic's? Or RSS?

From my previous Windows/Hyper V experience everyone and their brother recommended that TCP Off load options be turned OFF as it could cause performance issues with iSCSI. That was for a physical Windows host and in the case of Hyper V VM's a "good idea" from most blog's I read back when we had Hyper V

Thanks for any input!

Re: vnic's for in guest iscsi

jzygmunt70 — Thu, 25 Feb 2016 15:25:41 GMT

We've had good luck with in-guest iSCSI utilizing tcp off load with the VMware pv nic. In tests with various settings we found the best performance leaving tcp offloads turned on and RSS on. I've heard the same myths you've heard. Everyone's first troubleshooting step seems to be to to turn off TCP off loads though I haven't personally ever run into a case where disabling TCP off loads has actually solved a problem. Perhaps someone else out there can share some experience on that. As an aside, I usually don't recommend using in-guest iSCSI unless you have a really good reason to do so.

Re: vnic's for in guest iscsi

lindy37 — Thu, 25 Feb 2016 15:44:14 GMT

Thanks Jonathan! I called support yesterday and they said to leave the defaults but to change the Jumbo frames setting if my physical NIC's on the VMware host are set to that. So I only changed the Jumbo frames setting.

I migrated 1TB of Exchange 2010 data from VMDK's on a Equallogic to these new volumes on the Nimble, presented via in guest iSCSI with no issues last night. A snap with verify ran great this morning peaking at 16k read IOPS with 95% cache hits and less than 1ms latency. I think the vnic settings are good to go

I am only using in guest iSCSI for SQL and Exchange servers that I want application consistent snap shots of. For Exchange it is the only way to clear the log files with a snapshot backup. I am not a huge fan of VMware snapshots, although version 6 supposedly greatly improved them.

Thanks again!

Re: vnic's for in guest iscsi

jzygmunt70 — Thu, 25 Feb 2016 21:02:00 GMT

We use hypervisor based iscsi for Exchange and SQL. We're using tools like Veeam which execute the snapshots and our backups are app consistent. One of the uber Nimble experts can chime in here, but as I understand it, the Nimble will call VMware when it does a snapshot which in turn calls the VMware tools on the guest....in turn calling VSS which tells exchange / sql to quiesce and then gets an app consistent snap. So I think you can achieve what you want without resorting to guest basted iscsi.

Re: vnic's for in guest iscsi

lindy37 — Thu, 25 Feb 2016 21:58:35 GMT

"Nimble will call VMware when it does a snapshot which in turn calls the VMware tools on the guest....in turn calling VSS which tells exchange / sql to quiesce and then gets an app consistent snap"

You said it well and that is why I dont want to do that, to many pieces/steps and I have had VMware snapshots that did not revert in the past. I have read that the VMware snapshots in version 6 is the same tech they recently moved to for vmotion making them better, but still it is too many steps for me. Snapshot Consolidation changes in vSphere 6.0 - CormacHogan.com

With in-guest iSCSI - Nimble calls the NCM client in the guest which triggers VSS in the guest and takes the application consistent snap. VMware has no idea it is going on and is not an additional layer to deal with.

According to this (page 8/9)

http://uploads.nimblestorage.com/wp-content/uploads/2015/05/22100749/bpg_nimble_storage_vmware_vsphere5.pdf

Snapping from the VMware layer down will not clear the logs on Exchange. Prior to Nimble we uses Backup Exec 2014 which could either do in guest backups or from the VMware level with VMware snapshots. If all goes well we will be retiring Backup Exec and using some thirds party tools (UFS Explorer and Lepide) to do restore's from Snap Shot backups. We have a second Nimble that we replicate to as well.