wierd traffic out switchports

[Big Tom Davis]

I am seeing some wierd traffic on our HP2650's. I think we might be seeing a couple of anomalies...

1. The mac table appears to be clearing.
2. Traffic destined for other ports is being sent out other ports...
3. Some combination of the above.

To replicated number 1, I set one of our switches to mac-age-time of 100
I had a script running on a telnet session that simply did a show mac ever 1 second..
Every 1/10 of the mac-age-time the mac table collapses. This replicates for any mac-age-time you want to pick...

To replicate number 2, I setup a test switch 2650 with a default configuration and put it on one of our busy vlans.
I then put a Laptop running wireshark (used to be ethereal) and configured the capture filter as below:
not broadcast and not multicast and not ether host XX:XX:XX:XX:XX:XX
Note: mac address XX:XX:XX:XX:XX:XX is the laptop in question.
What this should do is cause wireshark to capture any traffic that is not specifically destined for it other than broadcast or multicast...
What I am seeing is traffic that should not be going out the port in quesion. IE HTTP traffic between 2 other PC's, etc...

Has anyone else seen this?

[Mohieddin Kharnoub]

Hi

I've tried what you did with WireShark, it look a cool idea really, but ...
how can you sniff out and capture traffic without a monitoring port, you have a switch not a HUB , so any HTTP traffic from a PC passing your 2650 switch will go directly to the default gatway port, which is in the mac table.

Now if your MAC table is clearing very fast, then your 2650 turns simply to HUB, it broadcast all traffic to all ports, then you can enjoy with the WireShark :)

I have used 2600 series, and never heard of that before, i will suggest you as any one can does to upgrade the firmware.

Good Luck !!!

[Matt Hobbs]

Hi Big Tom,

Have a look through this thread:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1017485

The usual reason for this behavior is spanning-tree topology changes occurring too frequently on the network. Topology changes will cause switches participating in the spanning-tree to flush their mac-address table.

A switch will flood unicast traffic only if it does not know what port the destination mac-address belongs too. As you've already noted the mac-address table seems to be clearing.

One other situation that can cause unicast flooding is if the mac-address table becomes full.

Matt

[Big Tom Davis]

Thanks Mat, Yes I am familiar with spanning tree collapsing the mac table and that is the first place I looked. It is not a spanning tree issue. It is not an issue with mac table being full because I see the issue with a switch with only a few mac's.

[Matt Hobbs]

Seeing some unicast traffic being flooded on a network is normal as mac-addresses age out of tables. I wouldn't expect you to be able to see 2 way HTTP traffic though between two hosts on the same broadcast domain.

You may see some coming from the router, as it probably has an arp age table longer than the standard mac-address aging timer.

With the first part of your issue, the mac-address table seeming to clear without spanning-tree topology changes, what firmware are you running? Although I haven't heard of any similar bugs in the past, update to the latest if you haven't already. When you say 1/10, in your examples does that mean it would clear every 10 seconds instead of 100?

[Big Tom Davis]

Mat:
I don't think this is an "router age" thing because I am seeing a lot of traffic. I could understand a few packets but on a busy network I see 100's of packets. The amount of traffic seems to be tied directly to the mac table being dumped.

Your right about the mac-age-time relating to the mac table clearing. It is 1/10 of the mac-age-time ie. 300=every 30 seconds, 400=every 40 seconds etc.

I am running the latest code 08.98...

[Matt Hobbs]

I just tried my 2650 with H.08.98 and set mac-age-time 60... I have few hosts, mainly other switches so they're not the most talkative bunch:

First off I ping the broadcast address to get their attention, then I kept on show mac'ing for 60 seconds:

ProCurve Switch 2650# ping 10.0.0.255

ProCurve Switch 2650# show mac

Status and Counters - Port Address Table

MAC Address Located on Port
------------- ---------------
0001e6-f98e00 49
0002b3-a660b6 50
000883-06ed80 49
000883-cce400 49
001185-d05780 49
001185-d05793 49


It is only near the end of the 60 seconds that I see them start to drop off which I was I hoping. Certainly not at 6 seconds.

I am out of ideas right now. Usual suspect is spanning-tree, otherwise it can be normal depending on the amount of unknown addresses due to the normal mac-aging process.

You may want to open a call with HP support direct if you can reproduce to 1/10 clearing issue in a lab type setup.

If you'd like me to test anything else I'm more than happy to.

[Big Tom Davis]

I already have it escalated to 3rd level support so from an HP standpoint they are on it...

[Big Tom Davis]

Well the problem ended up being a spanning tree problem. One of my switches was sending Topology changes by the bunches. After HP support explained to me that when a switch recieves any TC it clears the mac table at 1/10 the mac-age-time I saw the connection. Additionally when you do a show span, the Time Since Last Change only relates to TC's which the switch in question is in the topology path. So a TC in a different path will not affect this. This means you might have a Time since last change of days, which I had, and the switch still clearing the mac table because it saw a TC from a neighbor. To actually see the "real" TC do a show span detail. Then you can see the incrementing TC's...