Switches, Hubs, and Modems
cancel
Showing results for 
Search instead for 
Did you mean: 

Hosts on Procurve switches receiving unwanted unicast traffics

SOLVED
Go to solution
Frederic Bret
Occasional Advisor

Hosts on Procurve switches receiving unwanted unicast traffics

Hi,
from time to time, many of my hosts receive unwanted unicast traffics whose destination host is not the good one, and can even be the plugged next to it on the swith and perfectly communicating with the world (IE : their arp entry has not expired in the switch arp/port cache)
From what we can see, most of the tcp packets are coming with P (Push) flash set, and icmp requests are propagated everywhere, even if the switch perfectly knows where to forward the packets...

In the following examples, neither 10.1.11.3 nor 10.1.11.94 lost their respective and mutual connectivity, but the third host did see the exchanges :
14:07:50.834222 10.1.11.3 > 10.1.11.94: icmp: echo request (id:0c08 seq:144) (ttl 64, id 64561, len 84)
14:07:51.835155 10.1.11.3 > 10.1.11.94: icmp: echo request (id:0c08 seq:145) (ttl 64, id 64566, len 84)

and in this one, none of packets should be seen by my third host :
15:02:23.570616 10.1.70.15.1843 > 10.1.10.24.143: . [tcp sum ok] ack 2482 win 16951 (DF) (ttl 123, id 12313, len 40)
15:02:24.220064 10.1.10.24.143 > 10.1.70.15.1844: P [tcp sum ok] 2342:2370(28) ack 813 win 5840 (DF) (ttl 64, id 16487, len 68)
15:02:24.242524 10.1.70.15.1844 > 10.1.10.24.143: P [tcp sum ok] 813:819(6) ack 2370 win 17004 (DF) (ttl 123, id 12318, len 46)
15:02:24.242790 10.1.10.24.143 > 10.1.70.15.1844: P [tcp sum ok] 2370:2393(23) ack 819 win 5840 (DF) (ttl 64, id 16488, len 63)
15:02:24.272370 10.1.70.15.1844 > 10.1.10.24.143: P [tcp sum ok] 819:829(10) ack 2393 win 16981 (DF) (ttl 123, id 12323, len 50)

It seems that my switches forget their arp/port associations and forward parts of established tcp traffics everywhere, just in case...

The entire procurve familly is implied in this (4000M, 2424M, 25xx, 26xx, 28xx). Of course, firmware versions are up to date. (in fact last firmware version supporting CDP for local administration convenience).

In this examples, the tcpdump used to detect these packets on the subnet is the following :
tcpdump -vni vlan1 src net 10.1.0.0/16 and dst net 10.1.0.0/16 and not host 10.1.0.255 and not broadcast
- I'm capturing unicasts traffics from and to hosts on the 10.1.0.0/16 subnet,
- 10.1.0.255 is the address of the listening host to exclude regular traffic to/from it.

Did you ever encountered things like that ?

Thanks in advance !
5 REPLIES
Paul Clayton_1
Frequent Advisor

Re: Hosts on Procurve switches receiving unwanted unicast traffics

Igot the same thing happening right now. I see all the traffic between two hosts on my port, and it is not broadcast traffic. Interestingly it is between a user from MSTC and a terminal services server. The traffic from this connection is blasting across all our swith ports and slowing the network down.
Sergej Gurenko
Trusted Contributor

Re: Hosts on Procurve switches receiving unwanted unicast traffics

Make sure you do not have Microsoft Servers running NLB services on the local segment.
Please include MAC addresses in the TCPDUMP output next time. I'm not sure the MAC addresses are unicast but not multicast.
Frederic Bret
Occasional Advisor

Re: Hosts on Procurve switches receiving unwanted unicast traffics

Hello Sergei,

You're right, I should have included -e in my tcpdump. You can trust me the frames that are flying around have unicast mac addresses. Right now the traffic has calmed down, but I still can grab packets :

19:27:51.276786 0:a:95:9e:d2:2a 0:b:db:d4:b4:99 0800 60: 10.1.11.65.139 > 10.1.11.94.2844: P [tcp sum ok] 102924733:102924737(4) ack 2555999541 win 64240 (DF) (ttl 64, id 4799, len 44)
19:32:32.171265 0:9:3d:0:d7:18 0:b:db:d4:b4:99 0800 104: 10.1.10.24.137 > 10.1.11.94.137: udp 62 (DF) (ttl 64, id 4943, len 90)
19:32:32.171415 0:9:3d:0:d7:18 0:b:db:d4:b4:99 0800 252: 10.1.10.24.138 > 10.1.11.94.138: udp 210 (DF) (ttl 64, id 30973, len 238)
19:32:52.558741 0:a:95:9e:d2:2a 0:b:db:d4:b4:99 0800 162: 10.1.11.65.139 > 10.1.11.94.2883: P 3190993873:3190993981(108) ack 2929788445 win 65212 (DF) (ttl 64, id 5795, len 148)
19:32:53.555843 0:a:95:9e:d2:2a 0:b:db:d4:b4:99 0800 60: 10.1.11.65.139 > 10.1.11.94.2883: . [tcp sum ok] ack 925 win 64288 (DF) (ttl 64, id 6357, len 40)


And source and destination addresses are well known unicast mac addresses :
> arp 10.1.11.65
sake (10.1.11.65) at 00:0a:95:9e:d2:2a on vlan1
> arp 10.1.11.94
wendy (10.1.11.94) at 00:0b:db:d4:b4:99 on vlan1

We don't have NLB load balancing on our network.

Frederic
Matt Hobbs
Honored Contributor
Solution

Re: Hosts on Procurve switches receiving unwanted unicast traffics

I'm sure there is some great explanation to this, just not sure what..

The only reasons I can think for flooding of unicast traffic are, 1. They've expired from the mac-address table on the switch (default 300 seconds), or 2. There are spanning-tree topology changes occuring which are flushing the mac-address tables.

On one of the newer switches, check your 'show span' to see when the last topology change was - if it's in the seconds or minutes then it's quite likely the problem is due to spanning-tree.

Couple of articles of interest below:

http://www.ciscopress.com/articles/article.asp?p=336872&rl=1

And: http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00801d0808.shtml

Frederic Bret
Occasional Advisor

Re: Hosts on Procurve switches receiving unwanted unicast traffics

Thank you Matt,

you're absolutely right, topology changes are almost permanent here, we did in fact noticed that but didn't know if it was a cause or an effect..
But topology changes are difficult to track on Procurves, unless you shutdown network branches to see if topology changes stop. That's what I did and some of the switches configured RSTP in STP compatibility mode were causing the instabilities. We now have no (or long time duration between) topology changes, and we will see in week activity if unicast flooding persists....

More informations to come...

Frederic