Comware Based
1752613 Members
4913 Online
108788 Solutions
New Discussion

VXLAN service-instance on interface

 
SOLVED
Go to solution
Hugo29
Occasional Advisor

VXLAN service-instance on interface

Hi,

We are migrating from legacy VLAN configuration to VXLANs and VSI.

As I found, using tha same interface for both, VLANs and VSI service instances can cause  problems.  VSI works fine, but vlans start to loose packets. Therefore we made two uplinks to switches - one for vlans and one for vxlans.

On one switch I found interesting problem. When I made service instance on one port and connected cable to uplink, i got cyclotron. Because it is in production system, I disconnected cable instantly and started to investigate.  Traffic from uplink is sent untagged.  Therefore service instance encapsulation is untagged too.  I found, that packets are still going to vlan, that is assigned as pvid, not to service instance vsi.

displ l2v serv in te 1/2/3 ser 456 ver i see all is ok.

Interface: XGE1/2/3
Service Instance: 456
Type : Manual
Encapsulation : untagged
Bandwidth : Unlimited
VSI Name : vsi456
Link ID : 2
State : Up
Statistics : Disabled

 

interface Ten-GigabitEthernet1/2/3
port link-mode bridge
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 2 
port trunk pvid vlan 2
#
service-instance 456
encapsulation untagged
xconnect vsi vsi456

displ mac-add vlan 2  shows a lot of MACs,  

disl l2v mac vsi vsi456 is empty.

 

Where can be a problem?

PS. When problem started, interface configuration was empty. I made these entries to avoid cyclotron with second uplink in default vlan 1. 

 

 

 

7 REPLIES 7
Ivan_B
HPE Pro

Re: VXLAN service-instance on interface

Hi @Hugo29 !

You didn't mention the model of your switch. It is important, because different models have different restrictions, for example 5930, 5940, 5950 have this one, which I believe is relevant to your case:

When you configure Ethernet service instances, follow these access mode restrictions:
• You must use Ethernet access mode if one of the following criteria is configured:
􀁻 encapsulation tagged
􀁻 encapsulation untagged
􀁻 encapsulation default
• You can use Ethernet access mode or VLAN access mode if any other criterion is configured.

Since the default VSI access mode is VLAN, the command "xconnect vsi vsi456" uses this mode. Instead, the command should look like "xconnect vsi vsi456 access-mode ethernet". But as I mentioned earlier, it may not be applicable to your platform, therefore it always helps when you mention model of the device and s/w version running.

 

I am an HPE employee

Accept or Kudo

Hugo29
Occasional Advisor

Re: VXLAN service-instance on interface

Hi,

Thank's for explanation.

Model is FlexFabric 5940 2-slot JH397A

SW version 7.1.070, Release 2612P08

 

I need to receive untagged traffic to vsi456 as in access mode. Similar to port acc vlan 456 or port trunk pvid 456 in legacy vlan configuration. 

This configuration allready works on other ports. To VirtualConnect cards I have "enc untagged" and it works.  To   other switches I have "enc svid something" and it works too.  There is no any q-in-q on this switch.

I really don't understand, what it means "access mode ethernet" in interface configuration, if I receive untagged packets..

 

Best regards,

Hugo29

 

Ivan_B
HPE Pro

Re: VXLAN service-instance on interface

Here is what documentation says about VSI access modes:

==========================================================
The access mode determines how a VTEP processes the 802.1Q VLAN tags in the inner Ethernet frames assigned to the VSI.
VLAN access mode—Ethernet frames received from or sent to the local site must contain 802.1Q VLAN tags.
􀁻 - For an Ethernet frame received from the local site, the VTEP removes all its 802.1Q VLAN tags before forwarding the frame.
    - For an Ethernet frame destined for the local site, the VTEP adds 802.1Q VLAN tags to the frame before forwarding the frame.
In VLAN access mode, VXLAN packets sent between VXLAN sites do not contain 802.1Q VLAN tags. VXLAN can provide Layer 2 connectivity for different 802.1Q VLANs between sites. You can use different 802.1Q VLANs to provide the same service in different sites.

Ethernet access mode—The VTEP does not process the 802.1Q VLAN tags of Ethernet frames received from or sent to the local site.
􀁻- For an Ethernet frame received from the local site, the VTEP forwards the frame with the 802.1Q VLAN tags intact.
􀁻- For an Ethernet frame destined for the local site, the VTEP forwards the frame without adding 802.1Q VLAN tags.
In Ethernet access mode, VXLAN packets sent between VXLAN sites contain 802.1Q VLAN tags. VXLAN cannot provide Layer 2 connectivity for different 802.1Q VLANs between sites. You must use the same 802.1Q VLAN to provide the same service between sites.
==========================================================

I agree with you that for the untagged traffic there is no much difference, but that is mentioned as a restriction and the word 'must' is there too. Nevertheless, I made a quick test in my lab and seems like VSI gets all those MACs traffic from which it receives on the port with service-instance configured. Even without "access-mode ethernet":

 

[5940-Ten-GigabitEthernet1/1/2]dis l2vpn mac vsi 456
MAC Address    State    VSI Name                        Link ID/Name    Aging
0016-c8aa-bb59 Dynamic  456                             XGE1/1/2        Aging
bcea-fa6f-70d0 Dynamic  456                             XGE1/1/2        Aging
--- 2 mac address(es) found  ---
[5940-Ten-GigabitEthernet1/1/2]dis mac-add vl 2
MAC Address      VLAN ID    State            Port/Nickname            Aging
<5940>dis l2vpn service-instance interface ten1/1/2 service-instance 456 verbose

Interface: XGE1/1/2
  Service Instance: 456
    Type          : Manual
    Encapsulation : untagged
    Bandwidth     : Unlimited
    VSI Name      : 456
    Link ID       : 0
    State         : Up
    Statistics    : Disabled
<5940>

[5940-Ten-GigabitEthernet1/1/2]dis th
#
interface Ten-GigabitEthernet1/1/2
 port link-mode bridge
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 2
 port trunk pvid vlan 2
 #
 service-instance 456
  encapsulation untagged
  xconnect vsi 456

 


Could you send me output of following commands:

 

display device slot 1 subslot 2
display device manuinfo slot 1 subslot 2 (!!! remove the serial number from the output !!!)
display interface ten1/2/3
display l2vpn vsi name vsi456 verbose
display mac-address interface ten1/2/3

 



Also, you've mentioned ports with VC connected that receive untagged frames, are those ports located on the same 5940 as this Ten1/2/3 in question? Could you send their configuration as well, maybe there is a difference...

 

I am an HPE employee

Accept or Kudo

Hugo29
Occasional Advisor

Re: VXLAN service-instance on interface

Hi,

Port number really was other, as You see, and vlan number too. I understand, that it means nothing, but, because I have NDA, I can't provide actual information. Port number now is real, because it's too hard to change it in slots and so on. 

Interesting, that in this switch it works on VC and direct server connections, but not with this switch. Oposite switch ports are untagged vlan 1 (default).  I can look at it's config, but I have no permition to configure something. 

 

Best regards,

Hugo29

 

 

<HP5940SW2>display device slot 2 subslot 2
Slot Subslot Type State Soft Ver Patch Ver
2 2 LSWMxxxxxx Normal 2612P08 None
<HP5940SW2>display device manuinfo slot 2 subslot 2
Subslot 2:
DEVICE_NAME : HP 5930 24p 10GBase-T and 2p QSFP+ Mod JH182A
DEVICE_SERIAL_NUMBER : CN90xxxxxx
MANUFACTURING_DATE : 2019-10-19
VENDOR_NAME : HP
<HP5940SW2>display interface ten2/2/4
Ten-GigabitEthernet2/2/4
Current state: UP
Line protocol state: UP
IP packet frame type: Ethernet II, hardware address: 4cae-xxxx-xx6c
Description: Ten-GigabitEthernet2/2/4 Interface
Bandwidth: 1000000 kbps
Loopback is not set
Media type is twisted pair
port hardware type is 10G_BASE_T
1000Mbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
Flow-control is not enabled
Maximum frame length: 10000
Allow jumbo frames to pass
Broadcast max-ratio: 100%
Multicast max-ratio: 100%
Unicast max-ratio: 100%
PVID: 2
MDI type: Automdix
Port link-type: Trunk
VLAN Passing: None
VLAN permitted: 3999
Trunk port encapsulation: IEEE 802.1q
Port priority: 0
Last link flapping: 22 hours 39 minutes 18 seconds
Last clearing of counters: Never
Peak input rate: 115016645 bytes/sec, at 2020-12-07 12:29:28
Peak output rate: 94 bytes/sec, at 2020-12-07 12:28:30
Last 300 second input: 215 packets/sec 50281 bytes/sec 0%
Last 300 second output: 0 packets/sec 12 bytes/sec 0%
Input (total): 251515978 packets, 58139707082 bytes
1501978 unicasts, 21896484 broadcasts, 228117515 multicasts, 0 pauses
Input (normal): 251515977 packets, - bytes
1501978 unicasts, 21896484 broadcasts, 228117515 multicasts, 0 pauses
Input: 1 input errors, 0 runts, 0 giants, 0 throttles
1 CRC, 0 frame, - overruns, 0 aborts
- ignored, - parity errors
Output (total): 3027 packets, 976028 bytes
0 unicasts, 0 broadcasts, 3027 multicasts, 0 pauses
Output (normal): 3027 packets, - bytes
0 unicasts, 0 broadcasts, 3027 multicasts, 0 pauses
Output: 0 output errors, - underruns, - buffer failures
0 aborts, 0 deferred, 0 collisions, 0 late collisions
0 lost carrier, - no carrier

<HP5940SW2>display l2vpn vsi name vsi456 verbose
<HP5940SW2>display l2vpn vsi name vsi456 verbose
VSI Name: vsi456
VSI Index : 6
VSI State : Up
MTU : 1500
Bandwidth : Unlimited
Broadcast Restrain : Unlimited
Multicast Restrain : Unlimited
Unknown Unicast Restrain: Unlimited
MAC Learning : Enabled
MAC Table Limit : -
MAC Learning rate : -
Drop Unknown : -
Flooding : Enabled
Statistics : Disabled
Gateway Interface : VSI-interface 456
VXLAN ID : -
ACs:
AC Link ID State Type
XGE1/2/5 srv456 0 Down Manual
BAGG105 srv456 1 Up Manual
XGE2/2/4 srv456 2 Up Manual
<HP5940SW2>display mac-address interface ten2/2/4
MAC Address VLAN ID State Port/Nickname Aging
<HP5940SW2>display mac-address interface ten2/2/4
MAC Address VLAN ID State Port/Nickname Aging
0009-xxxx-xxxx 2 Learned XGE2/2/4 Y
000c-xxxx-xxxx 2 Learned XGE2/2/4 Y
....... about 30
<HP5940SW2>

Hugo29
Occasional Advisor

Re: VXLAN service-instance on interface

Hi,

Can I really assign ethernet access mode just for this one interface only? I have no such an option:

[HP5940SW2-Ten-GigabitEthernet2/2/4-srv102]encapsulation untagged ?
<cr>

I have opened case for it. Number 5352095326. 

Best Regards,

Hugo29

 

 

Ivan_B
HPE Pro
Solution

Re: VXLAN service-instance on interface

Hi @Hugo29 !

Yeah, information in the case doesn't match initial description, but I understand, it's security measure.
I don't see any MACs in VLAN2 learned on Ten2/2/4 (I'm looking at the diag, the port is up) and the all the MACs coming from the Cisco SG300 are learned on Ten1/2/10. However, there are no learned MACs inside the VSI102 (display l2vpn mac-address), but the port 2/2/4 is up and vsi102 is up too, so again it's a little bit different from what you see now when executing "display" commands... However, if we take the situation in the diag as a baseline and ignore the fact you see mac addresses learned in vlan 2 on the Ten2/2/4, then I think I know what is the issue and I was able to reproduce it, thanks to the diag in the case - it gave me one important bit of information - "hardware-resource vxlan l3gw8k". 

Here is how I reproduced it and what is the solution:

 

[HPE-Ten-GigabitEthernet1/1/2]dis th
#
interface Ten-GigabitEthernet1/1/2
 port link-mode bridge
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3999
 port trunk pvid vlan 2
 shutdown
 #
 service-instance 1
  encapsulation untagged
  xconnect vsi vsi102
#
return
[HPE-Ten-GigabitEthernet1/1/2]dis curr conf vsi
#
vsi vsi102
 gateway vsi-interface 102
#
return
[HPE-Ten-GigabitEthernet1/1/2]dis curr int vsi102
#
interface Vsi-interface102
#
return
[HPE-Ten-GigabitEthernet1/1/2]dis l2vpn vsi
Total number of VSIs: 1, 0 up, 1 down, 0 admin down

VSI Name                        VSI Index       MTU    State
vsi102                          0               1500   Down

[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]undo shut
[HPE-Ten-GigabitEthernet1/1/2]%Jul  4 05:24:52:141 1976 HPE IFNET/3/PHY_UPDOWN: Physical state on the interface Ten-GigabitEthernet1/1/2 changed to up.
%Jul  4 05:24:52:142 1976 HPE IFNET/5/LINK_UPDOWN: Line protocol state on the interface Ten-GigabitEthernet1/1/2 changed to up.
%Jul  4 05:24:54:203 1976 HPE STP/6/STP_DETECTED_TC: Instance 0's port Ten-GigabitEthernet1/1/2 detected a topology change.
%Jul  4 05:24:54:208 1976 HPE STP/6/STP_NOTIFIED_TC: Instance 0's port Ten-GigabitEthernet1/1/2 was notified a topology change.
[HPE-Ten-GigabitEthernet1/1/2]%Jul  4 05:24:57:191 1976 HPE STP/6/STP_NOTIFIED_TC: Instance 0's port Ten-GigabitEthernet1/1/2 was notified a topology change.

[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]dis l2vpn vsi
Total number of VSIs: 1, 1 up, 0 down, 0 admin down

VSI Name                        VSI Index       MTU    State
vsi102                          0               1500   Up
[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]dis mac-address
MAC Address      VLAN ID    State            Port/Nickname            Aging
[HPE-Ten-GigabitEthernet1/1/2]dis l2vpn mac
[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]
[HPE-Ten-GigabitEthernet1/1/2]exit
[HPE]vsi vsi102
[HPE-vsi-vsi102]dis th
#
vsi vsi102
 gateway vsi-interface 102
#
return
[HPE-vsi-vsi102]
[HPE-vsi-vsi102] #### HERE COMES THE SOLUTION ####
[HPE-vsi-vsi102]
[HPE-vsi-vsi102]vxlan 102
[HPE-vsi-vsi102-vxlan-102]%Jul  4 05:25:40:130 1976 HPE IFNET/3/PHY_UPDOWN: Physical state on the interface Vsi-interface102 changed to up.
%Jul  4 05:25:40:131 1976 HPE IFNET/5/LINK_UPDOWN: Line protocol state on the interface Vsi-interface102 changed to up.

[HPE-vsi-vsi102-vxlan-102]
[HPE-vsi-vsi102-vxlan-102]exit
[HPE-vsi-vsi102]exit
[HPE]
[HPE]dis l2vpn mac
MAC Address    State    VSI Name                        Link ID/Name    Aging
bcea-fa6f-7099 Dynamic  vsi102                          XGE1/1/2        Aging
bcea-fa6f-70d0 Dynamic  vsi102                          XGE1/1/2        Aging
--- 2 mac address(es) found  ---
[HPE]
[HPE]dis mac-add
MAC Address      VLAN ID    State            Port/Nickname            Aging
[HPE]

 

 

Please, assign the vxlan to the vsi102 and if my lab test was correct you should see your MAC addresses learned inside the VSI.

 

I am an HPE employee

Accept or Kudo

Hugo29
Occasional Advisor

Re: VXLAN service-instance on interface

Hi,

Super! It works!

So simple! I even didn't imagine that there is some way, xvlan number is needed inside device. I thinked it's needed for vxlan tunnels, not for internal communication. 

Thank's!

Hugo29.