vSphere Network Beaconing and Virtual Connect (VC) discussion

[chuckk281]

Rick had a customer question regarding beacon probing:

************

A customer was asking about configuring vSphere hosts with network beaconing which can detect upstream switch.

Are there any know gotchas on this one or otherwise not advised to use with our VC modules ?

If HP has a whitepaper on using this, that would be helpful.

http://searchvmware.techtarget.com/tip/Basic-networking-options-for-VMware-ESX-on-blade-and-rack-servers

*************

Lots of discussion:

***************

From Chris K.:

This is only a guess but  you check  “Smart Link” in the VC SUS  hosting these vSwitches then you probably do NOT want use this _new_ beaconing feature in VMware also… it’s possible that the 2 could negatively impact one another during link state change…

If I were this customer I’d at least test how the 2 interact with one another – see if the link outages/times are affected with both on, only SmartLink on, and only vSphere beaconing on – use ping and note link outage times…

I’d say that using one or the other is sufficient.

From Hoa:

Smartlink performance under DR simulation condition never fail for my customers.  At an enterprise class Nexus  upgrade, a team of heavy weight Cisco team flew in to oversee the project was impressed by Smartlink as they tried but unable to bring a single host/VM down.    An elegant yet transparent solution.     With Smartlink, no more than 3 pings loss, without Smarklink 5-6 pings.

Reply from Rick:

I also believe vSphere’s beacon probing is doing more than what Smartlink does. Please correct my knowledge on smartlink if I’m off base.

Beacon Probling claims to do more than uplink (VC to next layer) switch failure detection. They claim to go hight up to another or more depending on the topology.

Example. If I pulled the cable between access layer (TOR) and distribution layer (EOR). Smart link would not flag that as an error or failure but beacon probing would alert of an issue on the network.

Back from Hoa:

Then let’s implement Beacon Probing if you so strongly believe in.     By the end of the day it’s between you and your customer.    

From Chris J.:

There is a practical argument that explains why beacon probing is not as useful as it sounds:

Beacon probing requires 3 NICs and independent triple infrastructure upstream to work well.  This is not how redundancy in most modern data centres networks is built.  Typical redundancy is 1 +1 with automatic and fairly transparent rerouting of Ethernet frames around the failed component.

Even when configured with 3 NICs at the ESXi server, sooner or later beacon probing hits the part of physical infrastructure that is dual redundant only.  This means that at least two beacons travel out through one of the components.  If this component fails you do not get return beacons. So this is not better that Link State Tracking.

Classical example would be when you use a typical configuration with two FlexFabric Modules.  You can have 3 NICs in the ESXi server vSwitch and you can configure 3 SUSes, but two of these SUSes will be using uplinks from the same FlexFabric.  If this FlexFabric module goes down your beacon probing will not be able to figure out which link is really affected.

I think that ESXi servers should only worry about first hop, where Smart Link or Link State Tracking help and leave the rest to the network infrastructure itself.

And info from Dan:

I would suggest reading more online about Beacon Probing.

From memory, it works best with 3 NICs on the same VLAN.  Something you are not likely to encounter in a VC world because any 1 VLAN can only exist once each on LOM 1 and 2.  Unless you have Mezz cards or full height blades with 4 LOMs.

With only 2 NICs, when a Beacon is sent from NIC 1 and never received from NIC 2, how does VMware know whether the transmit failed or the receive failed?

As such it probably has limited usefulness in a VC world.

That being said, I have to disagree with Chris slightly (Playing Devil’s advocate here) in that I was recently made aware that if you have 10 VLANs in a Multiple Networks set in a profile, and 9 of them come from SUS1_A but the 10^th comes from SUS2_A (different uplinks, same VC module), even if all 9 VLANs under 1A have Smart Link triggered due to all the uplinks of 1A failing, because the 10^th did NOT trigger SmartLink (it’s on 2A and its links are still up), the NIC port doing Multiple Networks does NOT go down and those 9 VLANs will be black-holed which is exactly what SmartLink was supposed to prevent.

Now this isn’t a very common config for customers I have worked with, but it does bring around a possible use case for Beacon Probing succeeding where SmartLink has failed.

And in the opposite case, SmartLink would take down the NIC ports faster than Beacon Probing would be able to detect the failed path so you would likely not have any major issues there.  Would be no different than a switch that has hung, stops passing traffic, and shortly thereafter reboots which drops the Link state on all its links for a while during the reboot.

*************

Comments?