Max number of switches/ports on Brocade Core-Edge Fabric

[white221g]

Hello Folks,
1) What is the maximum number of "PORTS" on a Brocade Core-edge 4gbps fabric with 48K as core and 4900 as edges?. How is this limit determined?. What happens if this limit is exceeded?

2)What is the maximum number of "SWITCHES" on a Brocade Core-edge 4gbps fabric with 48K as core and 4900 as edges?. How is this limit determined?. What happens if this limit is exceeded?

Is the answer to one of the above questions derived from the other? Please respond. Appreciate your help.

[krusty]

Hi white221g,

This is not a easy answer, as it would come done to a "designers choice". The 48000 could fan out to 256 switches, if you ISL'd then all together and that would give you 63 ports per switch for devices. So the answer would be over 16,000 ports. However HP only supports a maximum of 239 switches in a fabric, so now we are down to about 15000 ports. Neither of these would be a practical or supported SAN. As a matter of fact, I just checked the SAN Design Guide, and HP only supports a maximum of 2560 ports in a Brocade fabric (see page 105 - Rule #5)

Now, for a "real world" SAN, you have to take into account many things, such as how many devices are storage and how many are hosts. Then you need to look are fan in / fan out ratios. Many SAN designers use a fan out ratio of 3:1 for I/O intentive workloads and 7-15:1 for low I/O workloads. (see page 44). If you are new to the SAN world, I would recommend reviewing the SAN Design Guide and possibly getting some training before you go designing a 2000 port fabric....

Here's the link to the SAN Design Guide:

http://www.hp.com/go/sandesignguide

Hope this Helps,

Curt

[white221g]

Curt,
Thanks a ton for the response. Few more questions to understand completely:
1)From what i understand is: There is no limit to the "number of switches" in a fabric, but the limit applies to the "number of ports" on the fabric. Am i correct?

2)How did HP determine the limit(maximum of 2560 ports in a Brocade fabric)?. When technically we can have much higher number ports in the fabric, what would degrade if this limit is exceeded?.

[krusty · Accepted solution]

There is a technical limit of the number of switches in a fabric. You can only have I think 239 domains in a Brocade fabric. If each switch consumes one domain, that is 239 switches. BUT we can use FC-AL loops to get around that issue. Also on the new Blade switches we can use NPIV which allows more than one N-Port Identifier (how the devices are addressed in the fabric) per fabric port, so that also has an effect. I don't recall the theoretical limit of the FC protocol, but I believe it's in the millions of addresses, so we won't go there.

I reality, I think some of the largest fabrics in use today are in the 3000-4000 port port range (check me if anyone else knows of larger ones).

As for how HP determined the their limit I cannot answer for them. My guess is it's the largest they built in a lab so they could test it. (Truth in adverstising - I am an HP employee) I know you can exceed this number and it would work, it's basically a supportability question. If you exceed it and you SAN has issues, you would have to reduce it down to the maximum before they would escalate higher up the support chain.

Cheers,

Curt

[Uwe Zessin]

A fibre channel address is 24 bits giving us theoretically 16,777,216 addresses and split into 3 'fields':

domain.port.AL_PA (some descriptions use other names for the fields)

The 'domain' is 1..239 - some of the remaining numbers are used for managing the fabric. E.g. name server, time server, there is a range to give each FC switch unique address...

In many cases, the 'port' maps to the physical port address.

In the past, an N_Port was visible with AL_PA=0, although 0 is also used for the switch's FL_Port when a true loop is connected to a switch port.

When using NPIV (N_Port ID Virtualization), the different virtual WWNs are mapped to different values in the 'AL_PA' field.

There is also a 'trick' to get around the 239 domain limit:
- use a device called an N_Port aggregator -- that is kind of a switch, but it pretends to be one or more N_ports using NPIV when talking to other switches (FC switches usually from links with E(xpansion)_Ports)