topic Recommended Switches for CS210, CS215, CS300 in Array Setup and Networking

Recommended Switches for CS210, CS215, CS300

jeberhardt42 — Tue, 16 Sep 2014 21:50:24 GMT

I was reaching out to see what everyone has been using for switches in their environments and trying to find one that performed well and was cost effective. We have been reviewing our hardware builds for a SAN install and thought that we should revisit the switch side and make sure that we're putting in a model that is known to work well in a production environment. I was reading through the forums and found some references to different Cisco and NetXtreme switches, but was looking for something that was known to work well with Nimble. We have put in HP Procurve 2920al switches on a few installs, which seem to work well and they're not overly expensive, but we're willing to make a change for something better. Thoughts?

Re: Recommended Switches for CS210, CS215, CS300

Daniel-san — Fri, 19 Sep 2014 00:44:54 GMT

Using HP 5406zl with 10G to Nimble and ESX hosts here without any issues. Works great, competitively priced, and lifetime support makes TCO very compelling.

Re: Recommended Switches for CS210, CS215, CS300

jherrera126 — Mon, 22 Sep 2014 13:05:48 GMT

Joe - There are many considerations to evaluate when choosing an iSCSI switch. I have listed those considerations below. If the switch you are considering meets the items on the list then it should be good as an iSCSI switch. The main consideration is the buffer space per port, if that is low then you will have performance issues.

iSCSI Switch Features and Considerations:

• Level 2 or preferably level 3 managed switch

• 512K per port dedicated port buffer space

• Supports flow control. It should be turned on

• Supports Jumbo frame - needs to be configured on everything in the data path, use at customers discretion.

• Spanning tree. It should be turned off

• Multicast filtering. It should be turned off

• IP precedence. It should be turned off.

• Disable unicast storm control on each switch handling the iSCSI traffic. However, the use of broadcast and multicast storm control is recommended.

Re: Recommended Switches for CS210, CS215, CS300

chaynes41 — Mon, 22 Sep 2014 21:12:03 GMT

We've had good success with Cisco 3850s and HP 3800s, using 10G ports.

- Chris

Re: Recommended Switches for CS210, CS215, CS300

BenLoveday — Tue, 23 Sep 2014 01:49:06 GMT

Hey Chris,

How have the 3850s been for you? Are you running the latest 2.1.4 firmware with them? We have had some strange flow control issues when using the 3850s (with other vendor arrays) so keen to hear your experiences.

Cheers,

Ben

Re: Recommended Switches for CS210, CS215, CS300

chaynes41 — Tue, 23 Sep 2014 18:45:02 GMT

Been working well for us. We've only used Nimble storage for VMware ESXi hosts so far. We enable Flow Control on the storage ports and follow all the other Nimble recommendations that Jason mentioned above. We haven't had any issues so far. We are a Nimble Partner so we have several different customers running different versions of IOS. The last Cisco 3850 we setup is running IOS version 03.02.03.SE RELEASE SOFTWARE (fc2). We also use HP 3800 switches on occasion and have not had any issues with those either. I'd recommend either one.

- Chris

Re: Recommended Switches for CS210, CS215, CS300

BenLoveday — Tue, 23 Sep 2014 22:04:46 GMT

Hey Chris,

Thanks mate, good to know. So you haven't seen any issues where the switch reports flow control as being off? We had some strange issues with HP Storevirtual and ESX where they were all set to flow control autoneg and the switches wouldnt negotiate flow control on. When we switched back to the old switches flow control auto negotiated on no probs. We have tried 3.06 (i believe), and had heard of a known flow control bug but thats about the extent of my knowledge unfortunately.

Thanks again!

Ben

Re: Recommended Switches for CS210, CS215, CS300

nick_furnell — Wed, 24 Sep 2014 16:12:13 GMT

We have just refreshed our two Data Centers and used two pairs of Brocade VDX 6730 - very nice SFP switches. We had some fun connecting them, mainly due to the new v4 firmware, which changed the command set from HP style to Cisco style (and it wiped everything off too). Fortunately the firmware update was at the point of installation, it would not have been much fun otherwise. Each pair is connected via 4 x 10Gb TwinAx, giving more than enough bandwidth and allows for someone stupid to pull out quite a few before things start slowing down. Interestingly, Brocade have just released a "Design & Build" validation Test based on this range of switches Data Center Solution, Storage-Validation Test: Brocade VCS Fabric and Nimble CS220G-X2 Flash Storage Array - Brocade Com… so close to my design, it's untrue (we are running HP Gen8's as well).

My two Nimble CS220G's are firmware 2.0.8 & 2.1.4 - unfortunately the 2.1.4 is reporting duplicate Management IP's on the LAN (we are not sure why), it could be completely unrelated. I haven't updated the second array yet, that will be the test I guess.

So far all good though, edge switches are Brocade ICX 6450's - these are so different from the CLI, that they could be from a different manufacturer!

Re: Recommended Switches for CS210, CS215, CS300

jeberhardt42 — Wed, 24 Sep 2014 20:59:04 GMT

Have you ever made adjustments on the number of queues to increase the buffer size for a set of ports? I was reading through one of the data sheets for an HP 3800 and found this.

Selectable Queue Configurations:

Allow you to increase performance by selecting the number of queues and associated memory buffering that best meets the requirements of your network applications

After reading some more on the HP 3800 series and also the Cisco 3850 series, it looks like each comes out of the box with 8 queues configured. There were some other references on the HP forums where people were saying that you could drop the number of queues down to 2 (cannot set it to 1) to increase performance. Just seeing if anyone has heard of changing this setting and if it actually made a difference in an iSCSI network by increasing the port buffer size.

Re: Recommended Switches for CS210, CS215, CS300

amb154586 — Thu, 25 Sep 2014 15:34:27 GMT

We are using a pair of Arista 7050s for both our iSCSI traffic as well as our server LAN traffic. We've been running this for about 9 months and I have no complaints.

Re: Recommended Switches for CS210, CS215, CS300

PokerMunkee — Sat, 17 Jan 2015 18:51:11 GMT

Any issues using a ProCurve 2920-24G switch for iSCSI and vMotion traffic for a CS300?

Servers and CS300 are 1GbE.

Saw someone post in another forum site that Nimble said not to use the 2920-24G due to buffer size. But the specs I see on this switch looks good to me?

Re: Recommended Switches for CS210, CS215, CS300

jliu79 — Wed, 04 Feb 2015 03:56:20 GMT

I'm using a Procurve 2910al-24G at our DR site with a Nimble CS220. I found the switch is not good enough probably due to the buffer size. If I do a big file copy on the nimble volume it slows down the whole switch to the point that I lose ping packets to the other computers on that switch. I'm thinking of replacing it with a 3600 series.

Re: Recommended Switches for CS210, CS215, CS300

jjones130 — Thu, 05 Feb 2015 14:16:12 GMT

I know this has already been marked as answered but we're using a pair of Cisco Nexus 3064T to power our datacenter these days and I can't be happier. These give me 10G copper out of 48 ports and then we are cross connecting with 2 bonded 20G twinax, with another 2 10G SFP for uplink to the distribution layer. If you can find your way around a Cisco switch already the leap to NX-OS isn't bad, and for the specific networking best practices from Nimble I've outlined their configuration here: Configuring Networking for Nimble-vSphere iSCSI - Koolaid.info

Re: Recommended Switches for CS210, CS215, CS300

stevewysocki26 — Thu, 05 Feb 2015 17:13:35 GMT

I don't know how large you are trying to go here, but I just have two 48 port 28xx HP Procurve 1Gig switches for both my iSCSI fabric and controller failover, and I couldn't be happier with the performance. In my shop, 10GB would be serious overkill.