QinQ configuration being provider and customer edge on same switch

[Joepske]

Hi,

I'm configuringing my coreswitches 2x HP5800 in the datacenter of my customer.

On the coreswitches there will be connected some Servers, Firewall's and connections. One of the connections is from Equinix datacenter, called Cloud Exchange. Via this fiber connection, we are going to connect to Microsft Azure, via the Microsoft Express route.

To make this possible, Equinix requires QinQ, where I can define the S-VLAN myselfe, this one is passed trough to Microsoft. For each microsoft service (Private / Public / office365) I can than assign an C-VLAN.

The question is, how can I have for Example, VLAN 200 as C-VLAN on the 5800 (and also on microsoft side) and than add the S-TAG (for eg VLAN1000) before sending it to the fiber-uplink.

Normally the provider add's the S-VLAN when I send my tagged frames into their port, but know in this specific configuration, my core will be provider and customer switch. The only solution I can think of, is to create a port wich add's the S-TAG, and than use a patch cable from for example interface g1/0/1 to g1/0/20 where g1/0/1 is a trunk port with vlan 200 and and g1/0/20 is an QinQ port wich add's the S-TAG vlan 1000. The uplink port to Equinix is than tagged as vlan 1000.. 

Does someone know how to do this without using a physical cable? 

[Mike_ES]

Hi,

Could you attach your switches current configs?

Michal

[Joepske]

Hi Michal,

In the current config, there's still no QinQ configuration active, because I'm stuck with this question.

Next month we will migrate to the datacenter and the configuration has to work. For now I'm using an temporary HP E3500 switch to test this setup, but it looks like this switch is very limited in QinQ configuration (have it in mixed mode) and have it configured with the loop cable. See attached drawing for this.

But I need to add an S-VLAN tag on a C-VLAN before sending it on the L2 fiber link....

So attached image is work-arround and not what I want.

[Mike_ES]

Joepske,

Some time ago I configured simple QinQ topology based on the 5800 as my IRF and Procurve vlans transparent transporting. When I started, I used this info:

http://datacenterfun.com/comware-configuring-qinq/

So, in my case  int the middle I put Procurve switch to simulate ISP core (access ports only!). As I remember, on the Procurve I had to configure only access vlans, but on the 5800 uplinks ports trunks were needed (with "qinq enable" command).

It was only staging setup  but tested and worked fine, so unfortunatelly I cannot put the congigs to you (I lost it).

Michal

[Joepske]

Hi Michal,

thanks for your reply.

The situation you describe, uses also a customer switch, connected to your 5800. In that particular situation, it is not that hard to create QinQ because the port on the 5800 connected to a customer switch, add's the S-tag (qinq). 

My question, is how can I do it, without a customer switch. The 5800 has de Customer VLAN's and has to add a QinQ S-VLAN tag before sending it on the fiber, connected to the other side.. And this without creating a fysical link attached to 2 port's on the same 5800... 

I attached a new Visio drawing to explain it better... Hope someone can answer.

[Mike_ES]

Ok, Thx for the drawing. It's really helpful :-)

In summary what you trying to do is to impose S-VLAN Tag (vlan 10000) for output direction toward your ISP right?

Not really sure, if QinQ in general can do it for outgoing interface in general...

But let's begin from scratch, here is the Cisco-based well explained tutorial of your case:

http://netcerts.net/q-in-q-tunneling/

Typically, ISP should do tunneling configuration on their edge switched ports.

Michal

[Joepske]

Michal,

thanks for your reply.

I understand that a typical scenario prescribes that the provider add's the S-TAG con the Customer-Edge device. In this case (Equinix datacenters and their Cloud-Exchange service) they don't do that.

They create a L2 link with Microsoft (Express route) and I define the S-TAG in the Equinix portal. So eventuallly I send my S-VLAN1000 into their CE-Switch. and Microsoft also has VLAN1000 defined on the Expres-route Circuit. At the end of the circuit I can create in MS Azure Virtual networks, / BGP connection points for diffrent netwerk types (one for the private Azure environment, one for the public (saas) environment and one for Office 365 as an environment) so there are a maximum of 3 VLAN's (C-VLAN's) transfering over the express route connection, embedded in S-VLAN 1000.

The downsite is, that Equinix expects me to send de S-VLAN (and thus do the QinQ config myself).

I Can solve this by adding 2 switches wich connect to the Equinix fiber (cloud exchange) and call them the Provider-Customer-edge switches. But why should I add 2 extra switches in the rack, while being able to do this all on the same coreswitches... 

I think it aint possible to do the tunneling on the same switch without using an physical patch cable to do the trick. (illustrated below)

[Mike_ES]

OK, good drawing, again! :-)

Your developed QinQ workaround is just fine, but this is returned confirmation of my previous statement:

To encapsulate your production vlans into transport VLAN (S-VLAN, ID 1000) you can do it only for INPUT L2 interface, and you are trying to find solution to configure such thing using output switch interface. I don't know if it possible.

Anyone could advise?

Br,

Michal

[tdeserranno]

Did you try to simulate in HCL Comware simulator ?

As I'm facing a similar demand from one of my customers, I have quickly tried to simulate it.

I intended to do this with HP 5510 HI switches which I have at my customer as edge switch

See below the setup I created in HCL.  I patched gi1/0/2 and gi1/0/3 on each of the switches.

Configurations of both Cust edge and MSAzure edge switches

===========================================================================================

<Cust>dis cur
#
 version 7.1.059, Alpha 7159
#
 sysname Cust
#
 irf mac-address persistent timer
 irf auto-update enable
 undo irf link-delay
 irf member 1 priority 1
#
 lldp global enable
#
 system-working-mode standard
 xbar load-single
 password-recovery enable
 lpu-type f-series
#
vlan 1
#
vlan 200 to 201
#
vlan 1000
#
 stp global enable
#
interface NULL0
#
interface Vlan-interface200
 ip address 192.168.0.1 255.255.255.252
#
interface Vlan-interface201
 ip address 192.168.1.1 255.255.255.252
#
interface FortyGigE1/0/53
 port link-mode bridge
#
interface FortyGigE1/0/54
 port link-mode bridge
#
interface GigabitEthernet1/0/1
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan 1 1000
 combo enable fiber
#
interface GigabitEthernet1/0/2
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan 1 200 to 201
 combo enable fiber
#
interface GigabitEthernet1/0/3
 port link-mode bridge
 port access vlan 1000
 qinq enable
 combo enable fiber
 undo stp enable
#

=========================================================================================
 sysname MSAzure
#
 irf mac-address persistent timer
 irf auto-update enable
 undo irf link-delay
 irf member 1 priority 1
#
 lldp global enable
#
 system-working-mode standard
 xbar load-single
 password-recovery enable
 lpu-type f-series
#
vlan 1
#
vlan 200 to 201
#
vlan 1000
#
 stp global enable
#
interface NULL0
#
interface Vlan-interface200
 ip address 192.168.0.2 255.255.255.252
#
interface Vlan-interface201
 ip address 192.168.1.2 255.255.255.252
#
interface FortyGigE1/0/53
 port link-mode bridge
#
interface FortyGigE1/0/54
 port link-mode bridge
#
interface GigabitEthernet1/0/1
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan 1 1000
 combo enable fiber
#
interface GigabitEthernet1/0/2
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan 1 200 to 201
 combo enable fiber
#
interface GigabitEthernet1/0/3
 port link-mode bridge
 port access vlan 1000
 combo enable fiber
 undo stp enable

UNFORTUNATELY I CAN'T PING FROM ONE VLAN INT 200 or 201 to the other side ...yet.

Anyone suggestions or comments ?

[Mike_ES]

@tdeserranno,

I tested HLC lab but cannot setup port for IRF membership on the S5820V2 - do you had chance to configure it?

My output:

<H3C>dis irf
MemberID    Role    Priority  CPU-Mac         Description
 *+1        Master  32        182d-bfc9-0400  ---
--------------------------------------------------
 * indicates the device is the master.
 + indicates the device through which the user logs in.

 The Bridge MAC of the IRF is: 182d-bfc9-0400
 Auto upgrade                : yes
 Mac persistent              : 6 min
 Domain ID                   : 100

[H3C]irf-port 1 ?
              ^
 % Wrong parameter found at '^' position.
[H3C]irf-port 1

Michal

[tdeserranno]

I did not setup any IRF stack in my simulation.

Just took two switches, linked them through an 1G on their Gi1/0/1 ports and configured the vlans and qinq.

And finally made a connection on each switch between the gi1/0/2 and gi1/0/3 port.  (you have to do this connection in HCL using 'adding a 'manual' link'

[Mike_ES]

Yes, understood your topology. My question was little out-of-scope of this tread :-), because I haven't much experience with HLC. Now I testing it and check usefulness for my HPN cases.

Anyway, backing to the problem maybe it would be worth to create Edge switches in the path (just to simulate provider switches and full path) for testing if HLC switches properly imitates QinQ config with Comware 7?

Br,

Michal

[Joepske]

Hi Michal, Of topic, yes you can setup IRF in HLC: I have 2 switches running in IRF.

irf member 1 priority 1
irf member 2 priority 1

irf-port 1/1
port group interface Ten-GigabitEthernet1/0/49
port group interface Ten-GigabitEthernet1/0/50
#
irf-port 2/2
port group interface Ten-GigabitEthernet2/0/49
port group interface Ten-GigabitEthernet2/0/50

[Mike_ES]

Ok now IRF is working, sorry to all for off-topic ;-)

thx

Michal

[tdeserranno]

I doubt whether QinQ will work in the HCL simulator ?  I can't get it to work.

I have experienced that other layer 2 functionalities do not work in HCL while they work on real HW

For instance, applying L3 ACL's as packet-filter on a L2 interface does not work in HCL

For instance, applying a VLAN QOS policy to a VLAN does not work in HCL.

Will have to look for HW !

[Joepske]

Hi Tdesserrano

First of all, thanks for trying! 

I tried it myselfe too, struggeling arround with the simulator, and also hoping / guessing this is an simulator issue and not a real deal :) Just can't get QinQ to work.

I tried several things, also with a 'provider switch in between. But no luck to get it to work. So I will test it on the hardware itselfe.

I also notices that the command

display mac-address

doesn't show any entry's. Also an simulator issue? And maybe causing QinQ to fail.

Anyway, below a screenshot of the HCL layout.

[tdeserranno]

Indeed, no mac-address table entries in HCL.  We noticed this too.

I guess we'll have to simulate on HW.  Tomorrow I'll have hands on a couple of HP 5500 HI to test with.  I hope to be able to see at least a minimal QinQ testbed working.

I'll keep you posted.

[tdeserranno]

I setup the following test with two HP 5500 HI switches

Ping from VLAN 200 at one side to VLAN 200 at the other side worked.  Between the two sites, VLAN 200 was QinQ'ed over SVLAN 1000.

Configuration and pings+display commands attached.

[Joepske · Accepted solution]

Hi,

Thanks for the testing and setting it up. 

As told, I run the temporary environment on an HP 3500-24G switch, wich with the latest firmware also supports QinQ / BGP etc... I Thought, let me share these 'strange' settings with you. Because, when setting it up on the same switch with a looped cable as descibed above, you need to change the Layer3 MAC address on the VLAN that crosses QinQ.

Article here: http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=mmr_kc-0131185&sp4ts.oid=3437443

Extra changes to be made:

Within VLAN200 (Customer VLAN) I added:

ip-recv-mac-address 223344-223344

On the Customer QinQ port (on wich the looped cable is connected, I added the commands:

unknown-vlans disable
qinq port-type customer-network
untagged svlan 1000

On the Serviceprovider QinQ port (on wich the uplink to MS Azure is connected) I added the commands:

unknown-vlans disable
QinQ port-type Provider-network (which is default and not visible)
tagged svlan 1000

And so QinQ with a Looped cable on the same switch works on a HP3500 switch also, with above adjustments.

In the weekend of the 25th of March, I'll be migrating the customer, and the 5800's will be connected and configured. So fingers crossed that I don't need an L3 MAC RECV command (which doens't excists on the comware switches, but possibly could be replaced by the IP Source Binding <IP> <MAC> command?)

But overall conclusion for this post is, that it is NOT POSSIBLE to add an S-VLAN tag to an C-VLAN on the same switch WITHOUT using a looped cable construction.

Case closed!

Joep