Howto Connect Sites 5406zl

[mcintoshs]

I recently created an inter-site 10Gbps 802.1q VLAN connection between two sites terminated on HP 5406zl SFP+ module ports, let’s call the sites A (central site) and B (remote site) and it works fine.

Details:

Site A: 

LAN: 10.0.0.0 255.255.0.0

Inter-site VLAN 3998: 192.168.1.46 255.255.255.252

10Gbps port F1 tagged in VLAN 3998 and ‘no untagged’ in default VLAN

IP route 10.40.0.0 255.255.0.0 192.168.1.45

Site B:

LAN: 10.40.0.0 255.255.0.0

Inter-site VLAN 3998: 192.168.1.45 255.255.255.252

10Gbps port A1 tagged in VLAN 3998 and ‘no untagged’ in default VLAN

IP default-gateway 192.168.1.46 255.255.255.252

Now I have to create another 10Gbps connection between the remote site B and another remote site C. Sites A, B and C will essentially be in a bus topology. Most of the traffic flow is between Site A and Site B and between Site A and Site C. Very little if any traffic will flow between Site B and Site C. I am trying to figure out how to do this. Would this scenario work?:

Site A:

LAN: 10.0.0.0 255.255.0.0

Inter-site VLAN 3992: no ip address (or should this have an IP address of 192.168.0.3 within an expanded 29-bit network to which the other 2 remote sites would also belong?)

Ip route 10.10.0.0 255.255.0.0 192.168.0.2 (or should this address be the remote end IP address of the inter-site connection between Site A and Site B – 192.168.1.45?)

IP route 10.40.0.0 255.255.0.0 192.168.1.45

Site B:

LAN: 10.40.0.0 255.255.0.0

Inter-site VLAN 3992: 192.168.0.2 255.255.255.252

10Gbps port A2 tagged in VLAN 3992 and ‘no untagged’ in default VLAN

IP default-gateway 192.168.1.46 255.255.255.252

Ip route 10.10.0.0 255.255.0.0 192.168.0.1

Site C (only direct-connected to Site B):

LAN: 10.10.0.0 255.255.0.0

Inter-site VLAN 3992: 192.168.0.1 255.255.255.252

10Gbps port A1 tagged in VLAN 3998 and ‘no untagged’ in default VLAN

IP default-gateway 192.168.0.2 255.255.255.252

The gist of my question is whether there also needs to be a VLAN 3992 created on the Site A switch at all (since the 10Gbps connection is not direct-connected to Site A) and, if so, whether it needs an IP address and, if so, whether it needs to be within the same mini-network which inter-connects Sites B and C.

[paulgear · Accepted solution]

Hi!

If your connections are A<->B and B<->C, then the simplest way to set this up is to add a point-to-point VLAN between B & C just like the one between A & B, and enable routing on site B.  I would recommend moving from static to dynamic routing in conjunction with this, although if you're not expecting to grow any further then static will still work.  I prefer this setup because it most closely maps the underlying topology.

Here are some (untested) config snippets for this scenario:

switch A:

ip route 10.10.0.0 255.255.0.0 192.168.1.45

switch B:

vlan 3997

    untagged A2

    ip address 192.168.2.45 255.255.255.0

...

ip routing

ip route 0.0.0.0 0.0.0.0 192.168.1.46

ip route 10.10.0.0 255.255.0.0 192.168.2.47

switch C:

vlan 3997

    untagged A1

    ip address 192.168.2.47 255.255.255.0

...

ip routing

ip route 0.0.0.0 0.0.0.0 192.168.1.46

The scenario you've described will also work; as long as A[F1], B[A1], B[A2], and C[A1] are all in the same VLAN and all have a reachable IP address on that VLAN, then they should be able to route via one another.  If you find it easier to think about the two links as a bus rather than two point-to-point links, then you could go with this topology.  Sample configs:

switch A:

vlan 3998

    untagged F1

    ip address 192.168.1.46 255.255.255.0

...

ip routing

ip route 10.10.0.0 255.255.0.0 192.168.1.47

ip route 10.40.0.0 255.255.0.0 192.168.1.45

switch B:

vlan 3998

    untagged A1,A2

    ip address 192.168.1.45 255.255.255.0

...

ip routing

ip route 0.0.0.0 0.0.0.0 192.168.1.46

ip route 10.10.0.0 255.255.0.0 192.168.2.47

switch C:

vlan 3998

    untagged A1

    ip address 192.168.1.47 255.255.255.0

...

ip routing

ip route 0.0.0.0 0.0.0.0 192.168.1.46

ip route 10.40.0.0 255.255.0.0 192.168.1.45

I think an important thing to consider here is: how will your network look if you need to add a 4th switch?  Will the scenario look different depending on whether you connect switch D to A, B, or C?  How will the configurations look?  Given those ideas, having a bus topology probably makes sense, but the switch routing configs will become more and more complex the more switches you add, hence my suggestion to move to dynamic routing.

Also, take the time now to plan out how your IP addressing scheme might look with more switches on more different sites.  Allow a full /24 for every point-to-point link or bus, even if you don't use them.  Since you are using RFC1918 addresses, you have space to burn (and when IPv6 comes, you'll have even more space to burn).  Plan it so that things fit together logically (i recommend having some sort of fixed mapping between VLAN numbers and address ranges, e.g. VLAN 1010 == 10.10.0.0/16), and so that the addresses are easy to aggregate for routing efficiency (i.e. keep all of 10.10.0.0/16 on one site, all of 10.40.0.0/16 on another site, etc.).

Hope that all makes sense!