Inter VLAN Routing - unable to ping accross subnets

[Kishadee14]

I have an issue routing between vlans. I am using a router-on-a stick setup. I have 2- 3500yl switches with distributive trunking setup and then 3 HP2530 connected to it. Here is my config for the HP3500 (switch A) and one of the 2530's config followes.  I am trying to ping  192.168.120.11 (which is connected to the HP2530 )from swith A. I can't reach any vlans from the default LAN or from any other VLAN. Any help would be greatly appreciated.

Running configuration:

; J9310A Configuration Editor; Created on release #K.15.16.0005
; Ver #06:74.fc.7f.ff.35.ef:8e
hostname "3500A"
module 1 type j93xxa
trunk 1 trk1 trunk
trunk 3 trk3 lacp
trunk 4 trk4 dt-lacp
trunk 5 trk5 dt-lacp
trunk 6 trk6 dt-lacp
trunk 7 trk7 dt-lacp
trunk 8 trk8 dt-lacp
time daylight-time-rule continental-us-and-canada
time timezone -360
ip default-gateway 192.168.99.1
ip dns server-address priority 1 192.168.99.18
ip dns server-address priority 2 10.100.0.18
ip route 0.0.0.0 0.0.0.0 192.168.99.1
ip routing
switch-interconnect trk3
interface 3
   name "SWITCH-INTERCONNECT-SWITCH_B"
   exit
interface Trk4
   unknown-vlans disable
   exit
interface Trk5
   unknown-vlans disable
   exit
interface Trk6
   unknown-vlans disable
   exit
interface Trk7
   unknown-vlans disable
   exit
interface Trk8
   unknown-vlans disable
   exit
snmp-server community "public" unrestricted
snmp-server contact "Infrastructure Team" location "Tech Service -
 Manufacturing"
vlan 1
   name "DEFAULT_VLAN"
   no untagged 9-10,14-24
   untagged 2,11-13,Trk1,Trk3-Trk8
   ip address 192.168.99.24 255.255.255.0
   exit
vlan 20
   name "PEER_KEEPALIVE"
   ip address 10.99.99.21 255.255.255.0
   exit
vlan 97
   name "PRIMARY_WAN"
   untagged 21-24
   tagged Trk3
   no ip address
   exit
vlan 99
   name "MPLS_LAN"
   untagged 14-16
   tagged Trk3
   no ip address
   exit
vlan 100
   name "GUEST_WiFi"
   tagged 17-20,Trk3
   no ip address
   exit
vlan 101
   name "CORP_WiFi"
   tagged 17-20,Trk3
   no ip address
   exit
vlan 102
   name "WiFi_MGMT"
   untagged 17-20
   tagged Trk3
   no ip address
   exit
vlan 120
   name "MFG_SAN"
   tagged Trk1,Trk3
   ip address 192.168.120.3 255.255.255.0
   exit
vlan 121
   name "IT_LAB"
   tagged Trk1,Trk3
   no ip address
   exit
vlan 122
   name "VOICE_LAN"
   untagged 9-10
   tagged Trk1,Trk3
   ip address 192.168.122.5 255.255.255.0
   exit
spanning-tree
spanning-tree Trk1 priority 4
spanning-tree Trk3 priority 4
spanning-tree Trk4 priority 4 bpdu-filter
spanning-tree Trk5 priority 4 bpdu-filter
spanning-tree Trk6 priority 4 bpdu-filter
spanning-tree Trk7 priority 4 bpdu-filter
spanning-tree Trk8 priority 4 bpdu-filter
distributed-trunking peer-keepalive vlan 20
distributed-trunking peer-keepalive destination 10.99.99.22

HP2530-Switch 3

Running configuration:

; J9775A Configuration Editor; Created on release #YA.15.16.0005
; Ver #06:04.9c.6e.ff.35.27:11
hostname "MFG-HP2530-3"
trunk 47-48 trk8 lacp
ip default-gateway 192.168.99.1
snmp-server community "public" unrestricted
vlan 1
   name "DEFAULT_VLAN"
   no untagged 1-24
   untagged 25-46,49-52,Trk8
   ip address 192.168.99.4 255.255.255.0
   exit
vlan 120
   name "MFG_SAN"
   untagged 1-19,21-24
   tagged Trk8
   ip address 192.168.120.2 255.255.255.0
   exit
vlan 121
   name "IT_LAB"
   tagged Trk8
   no ip address
   exit
vlan 122
   name "VOICE"
   untagged 20
   tagged Trk8
   no ip address
   exit
spanning-tree
spanning-tree Trk8 priority 4
password manager

[Kishadee14]

Also, I haven't untagged any ports on vlans 121 or 122 yet on the HP2530.... I am trying to get vlan 120 working first.

[Vince-Whirlwind]

Layer3:

You will need to provide the following further information:

 - Switch B VLANs 1 and 120 IP addressing details

 - For each of these two subnets, the hosts' subnet mask and default gateway info

[Vince-Whirlwind]

Layer2:
I think you've probably got this right, but we need to see:

 - interface config of all physical interfaces providing connectivity between the two 3500s, as well as between the 3500s and the 2530s, plus the Trunk interface configuration for each of these trunks.

[Kishadee14]

LAYER 3
The router is a SonicWALL NSA 4600
and the X0 LAN interface is trunked and configured with vlans 1, 120, 121, and 122 :
Vlan 1 -192.168.99.1/24 - gw 192.168.99.1
Vlan 120 -192.168.120.1/24 --gw 192.168.120.1
Vlan 121 - 192.168.121.1/24 --gw 192.168.121.1
Vlan 122 - 192.168. 122.1/24 --gw 192.168.122.1

Host info VLAN 120
For the host on the 120 vlan that I am trying to reach:
IP 192.168.120.11/24
GW 192.168.120.1

Host Info VLAN 122
IP 192.168.122.75
GW 192.168.122.1


I have two SW NSA 4600 in an HA configuration and on each 3500 port 1 is trunked to the X0 interface of the SW.
Failover and redundancy works fine, its just the vlans can't pass traffic to each other.

Trunk 3 LACP is my switch interconnect (keepalive) for distributive trunking

Switch B's Config
3500B# sh run

Running configuration:

; J8692A Configuration Editor; Created on release #K.15.16.0005
; Ver #06:74.fc.7f.55.3f.ef:de
hostname "MFG_3500B"
module 1 type j86xxa
trunk 1 trk1 trunk
trunk 3 trk3 lacp
trunk 4 trk4 dt-lacp
trunk 5 trk5 dt-lacp
trunk 6 trk6 dt-lacp
trunk 7 trk7 dt-lacp
trunk 8 trk8 dt-lacp
ip default-gateway 192.168.99.1
ip dns server-address priority 1 192.168.99.18
ip dns server-address priority 2 10.100.0.18
ip route 0.0.0.0 0.0.0.0 192.168.99.1
ip routing
switch-interconnect trk3
interface 3
   name "SWITCH-INTERCONNECTION-SWITCH-A"
   exit
interface Trk4
   unknown-vlans disable
   exit
interface Trk5
   unknown-vlans disable
   exit
interface Trk6
   unknown-vlans disable
   exit
interface Trk7
   unknown-vlans disable
   exit
interface Trk8
   unknown-vlans disable
   exit
snmp-server community "public" unrestricted
vlan 1
   name "DEFAULT_VLAN"
   no untagged 9-10,14-24
   untagged 2,11-13,Trk1,Trk3-Trk8
   ip address 192.168.99.12 255.255.255.0
   exit
vlan 20
   name "PEER_KEEPALIVE"
   ip address 10.99.99.22 255.255.255.0
   exit
vlan 98
   name "SECONDARY_WAN"
   untagged 21-24
   tagged Trk3
   no ip address
   exit
vlan 99
   name "MPLS_LAN"
   untagged 14-16
   tagged Trk3
   no ip address
   exit
vlan 100
   name "GUEST_WiFi"
   tagged 17-20,Trk3
   no ip address
   exit
vlan 101
   name "CORP_WiFi"
   tagged 17-20,Trk3
   no ip address
   exit
vlan 102
   name "WiFi_MGMT"
   untagged 17-20
   tagged Trk3
   no ip address
   exit
vlan 120
   name "MFG_SAN"
   tagged Trk1,Trk3
   ip address 192.168.120.4 255.255.255.0
   exit
vlan 121
   name "IT_LAB"
   tagged Trk1,Trk3
   no ip address
   exit
vlan 122
   name "VOICE"
   untagged 9-10
   tagged Trk1,Trk3
   ip address 192.168.122.2 255.255.255.0
   exit
spanning-tree Trk1 priority 4
spanning-tree Trk3 priority 4
spanning-tree Trk4 priority 4 bpdu-filter
spanning-tree Trk5 priority 4 bpdu-filter
spanning-tree Trk6 priority 4 bpdu-filter
spanning-tree Trk7 priority 4 bpdu-filter
spanning-tree Trk8 priority 4 bpdu-filter
distributed-trunking peer-keepalive vlan 20
distributed-trunking peer-keepalive destination 10.99.99.21
password manager


3500B# sh lacp distributed

                              Distributed LACP

Local Port Status:

       LACP    Trunk   Port            LACP    Admin  Oper  
  Port Enabled Group   Status  Partner Status  Key    Key   
  ---- ------- ------- ------- ------- ------- ------ ------
  4    Active  Trk4    Up      Yes     Success 0      293   
  5    Active  Trk5    Up      Yes     Success 0      294   
  6    Active  Trk6    Up      Yes     Success 0      295   
  7    Active  Trk7    Down    No      Success 0      296   
  8    Active  Trk8    Up      Yes     Success 0      297   
 
Remote Port Status:

       LACP    Trunk   Port            LACP    Oper
  Port Enabled Group   Status  Partner Status  Key
  ---- ------- ------- ------- ------- ------- ------
  4    Active  Trk4    Up      Yes     Success 293   
  5    Active  Trk5    Up      Yes     Success 294   
  6    Active  Trk6    Up      Yes     Success 295   
  7    Active  Trk7    Down    No      Success 296   
  8    Active  Trk8    Up      Yes     Success 297   


3500B# sh trunks

 Load Balancing Method:  L3-based (default)

  Port | Name                             Type      | Group  Type    
  ---- + -------------------------------- --------- + ------ --------
  1    |                                  100/1000T | Trk1   Trunk   
  3    | SWITCH-INTERCONNECTION-SWITCH-A  100/1000T | Trk3   LACP    
  4    |                                  100/1000T | Trk4   dt-lacp
  5    |                                  100/1000T | Trk5   dt-lacp
  6    |                                  100/1000T | Trk6   dt-lacp
  7    |                                  100/1000T | Trk7   dt-lacp
  8    |                                  100/1000T | Trk8   dt-lacp

[Kishadee14]

On each of the  4 switches, I have created a lacp trunk group  - one running to each of the 3500s

Switch 3

HP2530-3# sh trunks

 Load Balancing Method:  L3-based (default)

  Port | Name                             Type      | Group Type    
  ---- + -------------------------------- --------- + ----- --------
  47   |                                  100/1000T | Trk8  LACP    
  48   |                                  100/1000T | Trk8  LACP    
 

port 47 goes to switch A, port 48 goes to switch B

Each of the other switches are configured in this manner, of course they just have different trunk groups (trk4, trk5, etc.) But this is how the switches all connect back to the 3500s.

[Vince-Whirlwind]

OK, so your hosts are trying to use your router to get to different subnets on the network.

(This makes me wonder what all those IP addresses are doing on the switches - why have you configured them on there?)

So, VLANs just don't "pass traffic to each other". 

Hosts within a VLAN communicate with each other at Layer2, using MAC addresses.

Hosts in different VLANs communicate with each other at Layer2, using IP addresses and a routed interface.

In this case, your hosts are trying to use the Sonicwall router.

I know nothing about Sonicwall routers, and I don't understand what the "gw" is that you list against each VLAN interface, but if you have any inter-VLAN communication issues, it's the router where it's (not) happening.

[Kishadee14]

GW = gateway

I put ip addresses on my vlan interfaces because without them I am not able to ping  any host in that particular vlan that is directly connected on that switch.

I  cannot ping from one switch to the next even within the same vlan, I don't think the router comes into play then. Seems as though vlan traffic does not pass the switch.

Are you able to validate my config?

vlan traffic is not leaving the switch, I just verified this with the vendor. We set up a packet capture and nothing from the vlans is making it to the router. 

[Vince-Whirlwind]

You initially said, "I have an issue routing between vlans".

Now you're saying "I  cannot ping from one switch to the next even within the same vlan".

If you try to troubleshoot the Layer3 issue  "I have an issue routing between vlans" when you have the Layer2 issue "I  cannot ping from one switch to the next even within the same vlan", you won't get anything fixed.

Ignore Layer3 and inter-VLAN routing and address the Layer2 issue first.

I'd remove all those IP addresses from the switches - they will only confuse matters.

I'd delete all the trunks and LACP config.

Start off by patching the 2530 into one of the 3500s.

Configure the interfaces at both ends of the connection as "VLAN1 untagged" with no other VLANs on it.

Now put a host on a "VLAN1 untagged" interface on each of the two switches, give each of those hosts an IP address in the VLAN1 subnet and get them to ping each other.

Only after the above is working, add VLAN120 as a tagged VLAN to the interfaces on either side of the link between the 2530 and the 3500, change both the hosts' switch interfaces to "VLAN120 untagged" and get them to ping each other.

If you have that working, then you've got your VLAN trunking working OK, you can now create your distributed trunk, then repeat both tests to ensure VLAN1 and VLAN120 are still connected between the 2530 and the 3500s.

Only then, configure the trunk to the router with both VLANs 1 and 120 on it, either both tagged or 1 untagged, depending on the router config.

Cconfigure one of the hosts' switchports to VLAN1 untagged and get it to ping its default GW.

Reconfigure the IP address on the other host to an IP address that is in the VLAN120 subnet (leaving it on a VLAN120 untagged switchport) and get it to ping its default GW.

If both can ping their default GW, they will be able to ping each other. Job done.