VC Flex10 connect to Low End Procurve V1410 unmanaged switch as heartbeat LAN

[chuckk281]

Lots of discussion around the use of unmanaged switches with Virtual Connect.

Hop Hing had a customer question:

************************

Customer has a C3000 with 2x Flex10 modules. Each Flex10 module has 2x combine uplink ports (SUS) connected to customer data centre core switch (Cisco Catalyst) as production lan and another 2x combine uplink ports (SUS) connected to Low End Procurve V1410 unmanaged switch as heartbeat lan. The production lan is working fine, but the heartbeat lan is having network issue. Someone mentioned about network loop occurred in the heartbeat lan. The Procurve V1410 switch is a low end unmanaged switch without advance features like loop avoidance/prevention, LACP/trunking, etc. I know VC Ethernet has some sort of network loop prevention function.

The question is: can VC Flex10 combine uplink ports (SUS) be connected to this low end Procurve V1410 switch? Will it caused network loop, thus causing network issue?

***********************

From Robert:

The primary issue with trying to connect multiple ports of a VC net or SUS to an unmanaged switch is that VC will send periodic LLDP packets to the destination ports, and these  LLDP packets should never be forwarded beyond the destination port.  With an unmanaged switch, LLDP is forwarded to all ports, and received back on the other VC uplink port(s), causing VC to place the connection in stacking mode.  You will typically see the VC mac address appearing and disappearing in the ‘connected to’ field of the VC network or SUS.

From Mike:

First thing to check is why 2 of the Heartbeat-RJ45 have no link. It should have a link and status should be standby.

enc0:1:X3  OA1        Linked           SFP-RJ45  Auto   Heartbeat     
                      (Active) (1Gb)                                  
-----------------------------------------------------------------------
enc0:1:X4  OA1        Not Linked       SFP-RJ45  Auto   Heartbeat     
-----------------------------------------------------------------------
enc0:1:X5  OA1        Not Linked       absent    Auto   -- --         
-----------------------------------------------------------------------
enc0:1:X6  OA1        Not Linked       absent    Auto   -- --         
-----------------------------------------------------------------------
enc0:1:X7  OA1        Linked (10Gb)    HD        Auto   Stacking Link 
-----------------------------------------------------------------------
enc0:1:X8  OA1        Linked (10Gb)    HD        Auto   Stacking Link 
-----------------------------------------------------------------------
enc0:2:X1  OA1        Linked           SFP-RJ45  Auto   Production    
                      (Standby) (1Gb)                                 
-----------------------------------------------------------------------
enc0:2:X2  OA1        Linked           SFP-RJ45  Auto   Production    
                      (Standby) (1Gb)                                 
-----------------------------------------------------------------------
enc0:2:X3  OA1        Linked           SFP-RJ45  Auto   Heartbeat     
                      (Standby) (1Gb)                                 
-----------------------------------------------------------------------
enc0:2:X4  OA1        Not Linked       SFP-RJ45  Auto   Heartbeat     

Hop Hing replied:

Those 2x VC uplinks (2nd link on each of the Flex10 modules for Heartbeat) are causing the network issue and are physically disconnected now. From the BL860c i2 blade point of view, there is no APA (NIC teaming) configured.

Any idea or experience of the issue? Is Procurve V1410 switch supported with VC/VC Flex10? Thanks.

From Chris:

I don’t know about supported.

I use a cheap Trendnet switch at the house and if I try and use more than one port  from the switch in an uplink set it fails. I can make multiple uplink sets but each can be only one port. The switch does support Vlans but does not have LACP functionality. I have seen and heard of these issues at customer sites, they work fine with one port in an uplink set, but fail with multiple, they all go standby on the VC and if you have Smartlink enabled it will disconnect the Nic.

I don’t know the cure but I have seen the issue. Some ammo to use on the customer, they spent $50k at least for and blade enclosure and cannot spend a couple of thousand for a decent switch.  A customers spends a million on a disk array but will not cut a $75 tile for power, sometimes when you put things in perspective people see the error of their decisions.

From Dave:

–      Virtual Connect and Integrity Servers

–      HP-UX Auto-Port Aggregation

–      Virtual Connect is not a switch and therefore doesn't provide the underlying technologies required for Auto-Port Aggregation

• APA requires a port channel (Etherchannel or LACP) which VC does not support on the downlinks to the server NICs
• If you need to trunk multiple downlinks then you will not be able to do this using APA

When an Integrity server connects to Cisco switches in interconnect bays 1 and 2, and an uplink drops on a port (or trunked group) on the switch in bay 1 then APA can fail-over the NIC to enable the traffic to pass through an alternate port on bay 2 but that this capability depends on the down-links to the NICs being aggregated. This is not the same condition as when the uplinks from the BladeSystem switch to the core are aggregated. In c-Class, while VC does support port aggregation on the uplinks, it cannot aggregate signal from the NIC through the VC Domain to the uplink port.

Note: VC cannot aggregate ports on multiple modules (e.g. interconnect bay 1 port 3 + 4 with interconnect bay 2 port 3 + 4, but then neither can many edge switches).

Note: APA requires a port channel (Etherchannel or LACP) which VC does not support on the downlinks.

Note: If you need to trunk multiple downlinks (say for the Cache Fusion interconnect) then you will not be able to do this using APA.

From Hop Hing:

There is no VLAN defined or used in VC, only multi ports SUS is configured. There are 2x BL860c i2 blades in the C3000 enclosure and there is no APA (NIC teaming) configured in the blades. Is 802.1Q support still required? Or do we require a switch that supports LACP/trunking for multi ports SUS?

I am suspecting the multi ports SUS connection to V1410 is causing the issue. Your resolution below is what I have in mind: use single port uplink set or change the switch to one that has advance features like LACP/trunking/802.1Q/loop prevention, etc.

And from Lee:

We need to remember basic networking.  When multiple ports are created with a SUS, a network tunnel is being created.  For a network tunnel to work, both ends of the connection must support tunneling.  In this case, the VC module is trying to create the tunnel using LACP.  Your unmanaged switch is not able to create the other end of the tunnel because it does not support LACP.  To maintain network integrity, the VC module is going to disable all of the links in the network tunnel except one link so that it can maintain network connectivity.

An unmanaged switch is a cheap network connectivity solution and is meant to provide basic network connections between devices.  It is not designed for more sophisticated network designs such as tunneling or VLAN separation.  

************************

Any other comments or quesitons for the group on this subject?