VC Flex 10 and Cisco 6513 Switches

[aali]

Hello,

I have the following configuration

[Hongjun Ma]

the physically cabling may look like a loop but loop will not occur. The blocking mechnism happens on VC side. The first thing VC needs to do is to make sure no looping toward network side.

The reason 6513 is not blocking is that you should treat VC as a host ports, enable BPDU guard and STP edge for edge security.

The No.1 thing you need to remember about VC is that it doesn't talk STP with any switch at all. It uses its own local internal decision to block redundant path.

you can check my blog for some VC tech overview slides and other cookbooks.

[aali]

Thanks for your quick reply. 

Regards,

[aali]

Thanks

[Hongjun Ma]

can you describe more about "networking" flapping? do you see ports between 6513 and flex-10 going up/down? or you have intermittent vm connectivity issue on a particular vlan?

what's your flex-10 firmware version? did you open a HP support case so they can help you on VC side?

in regard to your idea about connecting two chassis together like the way you described? my answer is don't do this way.  by removing flex-10 module 2, in reality you are losing half of bandwidth for all blades. all blades 2nd port(1g or 10g) are internally mapped to flex-10 module 2. Also there are more issues you'll encounter like converting non-stacking design to stacking design. your life will be miserable. your current setup should ok perfectly fine and it's very common for c7000 setup so stay this way and resolve this "flap" issue first.

[aali]

Thank You so much!  The VC firmware is 3.30.

[Hongjun Ma]

it's good that you send out error message on c7000 side. From the c7000 log message, you actually have OA mgmt link flapping. This should have nothing to do with 6513 and VC. Usually you'll have a pair of mgmt switch to connect c7000 OA ports, this is pure for mgmt access of enclosures.

OA and VC are doing different things in c7000. OA being mgmt and VC being I/O control for user traffic.

so you need to check where OA link is connected to, are they going to any TOR switch, the problem is with the GE link between that switch and OA and then you need to check cables/transceivers.

In regard to your 6513 config, just enable STP portfast trunk and BPDU guard under port channels. Remember treat VC ports as "big" host trunk ports.

[aali]

I was incorrect about firmware version.  Actually right now the version is 3.18 and we will be upgrading to 3.51.

We actually do not see any network ports flapping at all. The 6513 ports do not show any errors at all. No intermittent connectivity on a single VLAN.

The Virtual connect is at version 3.18 - we will schedule an upgrade, presumably for Sunday the 25th to upgrade to version 3.51

There are just times when we lose connectivity to the Blade Enclosure based on the fact that we receive ICE alerts indicating VMS are unreachable. When we receive these messages, it has been noticed that some of the Physical servers - connected via copper to an entirely different blade on the 6513, will get connectivity errors. I can not correlate an exact time, but I believe that if the Switch CPU is getting to 90%, then the packets destined for those physical servers may be dropped.

Thank you so much.

[Hongjun Ma]

i'm a little confused. Do you have OA link flapping? I saw it  in the log message.

if 6513 is running 90%, "show proc cpu" should tell you which process is taking cpu cycles. even you run high on cpu, if you don't have any routing/mac table flapping, 6513 should still switch traffic without interruption because those are in H/W asic and not depend on CPU if there is no reprogramming from CPU.

3.18 is not very old. 3.51 will give you more features and is our current latest release.

[aali]

Sorry for the late reply as I was trying to get more details.  The problem is totally beyond my imaginations.  What we have learnt that this problem only happens every 5-6 day and almost around the same time.  According to OA Logs, it shows that gateway is unreacheable.  The wierd thing is that the ESXi hosts, Guest VMs don't go down at all it just stops working.  Looks like that the Cisco Switches are unable to send traffic between VLANs and even in the same VLAN.  As I said that this usually happens every 5-6 day around same time and then everything geos normal after 15-20 minutes.  I was suspecting that may be some time of jobs or Application running at that time is causing this behavior.  Since OA logs doesn't show any issues other than gateway is down or unreachable, HP VC Technical Support is suspecting that Cisco 6513 switches are causing this abnormal behavior.

I am running out of ideas except may be running network analyzer and capture the traffic when this event occurs.  We can also enable traffic shapping or rate-limiting on those VLANs so the traffic will not put too much borden on Cisco 6513 switches.  But again, I beleive Cisco Switche with Sup 720 are good enough to handle large amount of traffic.

Any help will be appreciated.

.