Tradeoff between Single Enclosure VC Domain and Multi Enclosure VC Domain

[chuckk281]

Girish was looking for some Virtual Connect and enclosure advice:

************

We are looking for stacking two enclosures in our datacenter. Can you please help us to understand the advantage of stacking two enclosures. Here is what we are looking to do.

Dual Ring 2 Enclosures

Note that uplinks are connected to alternate enclosures/VC modules (left side/right side). In addition to the external stacking cables (red lines) seen in the following graphic, the VC modules are also cross connected internally, creating a L2 ring topology within Virtual Connect.

This configuration provides the ability to sustain a chassis, module, uplink or upstream switch failure/outage and still maintain connectivity to the network. If additional bandwidth is required, uplinks could be added to these VC networks, or additional VC networks could be configured.

Here are questions from Girish and answers from Vincent

*****************

My current questions about this:

-          Can we do this with productive enclosures without downtime?

No, down time would be needed for the second enclosure.

One enclosure becomes the Domain Manager, while the second enclosure becomes subordinate.

You will need to delete the domain of one enclosure in order and clear VC mode in its OA to find and import it into the primary enclosure VC Domain

-          Do we need extra hardware e.g. network cards for enclosures?

Only the four DACs or 8 SFPs and four fiber cables.

Assuming that these two enclosure are matched VC modules including VC FC(if involved).

Looks like two Flex-10 modules with the CX-4 connectors.

-          Are there disadvantages? (e.g. less redundancy, dependence on other enclosure, lower bandwidtch,…)

Only be aware of firmware updates.

The entire Domain is updated during a FW update.

Be sure to have redundant paths or down time.

-          Is the only advantage to save 2 LAN ports (1 LAN port for each enclosure?) or is are there more benefits?

Internal traffic can pass through the stacking links to bypass an external switch.

All enclosure use one range of the WWN, MAC & SN .

VMotion can be passed from enclosure to enclosure through the stacking links.

-          If the benefits are not attractive enough and work and maintenance too much, we should not do it

-          effort/risks/disadvantages/advantages.

************

Other comments or suggestions?

[marcelkoedijk]

Your comments are correct :)

1 more disadvanced can be a have single VC domain, when the domain failed totaly it had effect on 32 server bays.

Please make sure to spit your cluster in minimum two separate vc domains. It all depends on the high availibilty that is required. The VC domain can be redundant configured but i see more than once that a fw bug in the software can bring down a hole vc domain.

Another small thing to keep in mind. enc0 will be the master enclosure, VCM is redundant in the master enclosure.

When the master enclosure enc0 failed, production will not involved, but VCM managent can not be reached.

VCM never can run in enc1 or above, just how its designed.