Recommended Server Configuration (with Two 10GB ports)

[J Benway]

wondering what the recommended configuration is for a server with two 10G ports running ESXi 5.1

thanks,jb

[benwatson87]

Round robin multipathing policy, with networking set as per VMware's best practices. There is a guide somewhere on it, I'll try to find it for you.

[benwatson87 · Accepted solution]

Actually, there is loads of useful documentation, lots on this site as it happens: https://connect.nimblestorage.com/content?filterID=all~objecttype~objecttype[document]&query=vmware

This is particularly interesting, if you are short on time: VMware vSphere on Nimble Best Practice Express Edition

I'd suggest also speaking to your local Nimble tech contacts, they'll be able to verify what you've done is correct.

[fbjocab111]

HI

Jason has given you good links.

And here are some point that may or may not apply to your environment.

• You need fault tolerance for ISCSi so you should use both cards for ISCSI in case of failure.(if possible some scenarios with binding may not do the trick)
• Also use iSCSI networking with multiple (v)Switches and ISCSI software, Binding, ISCSI Discovery and Round Robin.(see nimble iscsi install guide or jasons link)
• If you want to you can mixture with the depth of round robin there are articles in this forum for this or link observe that nimble recommends the value of 0 this should be verified in our environment depending of load and luns and so on, so be careful with these values they can show good values when doing test to a single machine or database but as a total load in our environment these values can change.
• Configure multi-NIC vMotion
• For best performance, use vmxnet3 virtual NICs.
• Configure all VMkernel network adapters to the same MTU. When several VMkernel network adapters are connected to vSphere distributed switches but have different MTUs configured, you might experience
  network connectivity problems.(jumbo frames is to prefer)
• Use vlan to separate traffic for the different networks, (clients, management ,iscsi and so on, Avoid default vlan if possible
• If using HA add extra isolation addresses for more secure environment. (das.isolationAddress)
• Try to make everything fault tolerant (iscsi, client, management switches cables and so on.)
• Verify that you pci slot in your server can do 10gb/s some have seen several cases where people have 10 Gb but can only reach 4 gb.

Here are some links that may help you more..

http://pubs.vmware.com/vsphere-50/index.jsp

http://virtualgeek.typepad.com/virtual_geek/2010/07/vsphere-41---what-do-the-vstorage-apis-for-array-integration-mean-to-you.html

http://defaultreasoning.wordpress.com/

http://www.windowsitpro.com/article/server-virtualization/multipath-iscsi-vsphere-5-141254

http://blogs.vmware.com/vsphere/2011/08/vsphere-50-storage-features-part-12-iscsi-multipathing-enhancements.html

http://vmtoday.com/2012/02/vsphere-5-networking-bug-affects-software-iscsi/

http://www.boche.net/blog/index.php/2011/09/28/changing-the-default-vsphere-5-0-psp-to-round-robin/

http://pubs.vmware.com/vsphere-50/index.jsp

[rocky36]

You may also want to read what Wen Yu posted back in April 2013 TO LACP or NOT to LACP for VM and iSCSI traffic

In this guide he shows how to use LACP for VM traffic and over the same physical ports use MPIO for the iSCSI traffic