Switches, Hubs, and Modems
1748201 Members
2976 Online
108759 Solutions
New Discussion юеВ

HP4000M flooding unicast traffic

 
SOLVED
Go to solution
Romeo Chance
Occasional Advisor

HP4000M flooding unicast traffic

I have been battling an ongoing intermittant
problem with the HP4000M switches where every few minutes we get ALL unicast traffic flooded on every active port.

We have the latest version of the firmware
installed and the problem still persists.

I noticed that there is a similar problem
with the newer switches as well.

basically the switch starts acting like
a hub every few minutes, i see all traffic
passing just as if the switch is a hub.

Monitoring Port is DISABLED, i installed
commview to see the traffic when i noticed
the flood of traffic on all ports.

13 REPLIES 13
Ron Kinner
Honored Contributor

Re: HP4000M flooding unicast traffic

Sounds like for some reason the Forwarding Database has gotten corrupted or erased since this behavior is normal for an empty Forwarding Database. Once a MAC is heard from it should be put into the FD along with its port number and then all unicast traffic to that MAC should be limited to its port number. Is there any sign that the swx is rebooting? Maybe in Show Version or Show Boot-history or show logging?

Perhaps a loop is driving the switch crazy? Do you have STP working on all ports to prevent loops?

Ron
Romeo Chance
Occasional Advisor

Re: HP4000M flooding unicast traffic

Ron,

Thanks for the reply,

1. No Loops - spanning tree is disabled
2. Only 4 devices on that vlan, all routers
3. Flood is not multicast, it is unicast
traffic (tcp/udp/icmp) between the routers
4. Switch does not reboot (up 32 days)
5. Flood comes every 3 or 4 minutes and
goes away after about 1-2 minutes

It is very strange, the MAC Age Period is
set to 5 Minutes on the switch (default).

When the flood appears, I cannot see anything
weird on the sniffer, all packets are correct
without any checksum errors and all packets
have valid source/dest mac-addresses.

thanks
Romeo

Ron Kinner
Honored Contributor

Re: HP4000M flooding unicast traffic

If it's only a few devices why not try converting them to static MACs. That might stablize the FDB.

If you don't want to do that you might try extending the age timeout and see if the time between floods follows the change.

Is there any chance that two of the devices have the same MAC? Easy to do with HPUX if you clone them.

STP is not your enemy. It's good to have it on if you have more than one switch.

Do you have a router involved here anywhere?

I suspect you may have a software bug tho it seems strange that someone else hasn't noticed it.

Ron
Ron Kinner
Honored Contributor

Re: HP4000M flooding unicast traffic

OOPS just reread your last post. Four routers. What kind? What dynamic routing protocol are you running? RIPV1, RIPV2, OSPF, EIGRP, IGRP?

Is it only this VLAN that has the problem?

Ron
Romeo Chance
Occasional Advisor

Re: HP4000M flooding unicast traffic

Ron,

OK 3 linux routers and 1 cisco router
with 3 paths to the internet.

Basically, I know its HP switch related
as I have today put a simple-simon
3com 10/100 switch just for this egress vlan
and it does not leak traffic.

I have another vlan on a seperate switch
which is having the same symptoms.

If the FDB is being corrupted how can
i check this or is there a newer firmware
in the pipeline to fix this problem.

btw, i have also disabled ABC and IGRP
just to see if it helped--- it doesnt.

thanks
Romeo




Ron Kinner
Honored Contributor
Solution

Re: HP4000M flooding unicast traffic

I would change the MAC age time to the max of 1000000 and see if that slows it down:

mac-age-time 10000000

The timer should be reset each time some traffic from a MAC passes through the switch but I suppose they might have messed up and not done that. That way after 5 minutes the whole table would age out about the same time.

There is a mechanism under security to set the the MAC addresses which are allowed on that port. You would think that if you told it that only one MAC was allowed and it was the MAC of the router that it would then not bother sending out stuff to other MACs. You might try that and see if that helps.

Also CDP has a 3 minute timer and it was recently added so you might try turning it off. I doubt you are using it for anything. I don't see it in the manual I downloaded but it was in the release notes. Think it should be something like:
no cdp enable

There is a note in the manual which says that duplicate MACs on different VLANS are not supported and can cause unspecified problems.

There is nothing in the problem reports but HP's site is really weak and only talks about two switches.

Ron

Mist Lifter
New Member

Re: HP4000M flooding unicast traffic

Hello,

All Ethernet switches, including the HP ProCurve 4000M, flood unicast traffic in the event that it has not yet learned the port number associated with a particular MAC address. Once the unicast is flooded out the ports, the response to that unicast is what the switch uses to learn the MAC address. Since MAC address learning is a relatively rapid process, I wouldn't expect to see the unicast traffic repeatedly flooded unless the switch did not have the opportunity to learn the MAC address from a reply (where the MAC address is listed in the source field), or the MAC address has timed out of the bridging table.

There are some specific instances in which the switch is unable to learn the MAC address. One example of this is when an imaging program is configured to use unicast traffic. Since the system targeted by the program generally can't respond (doesn't have a TCP/IP stack at that point), the switch is unable to learn the MAC address. Another issue with learning of MAC addresses is resolved in switches with updated firmware (C.09.16).

I hope this is helpful. If you believe that your switch is either a) not learning MAC addresses, or b) flooding unicast traffic for a MAC address that the switch knows, I recommend that you phone ProCurve technical support at (970)635-1000, options 3,2,2 for further troubleshooting.
Romeo Chance
Occasional Advisor

Re: HP4000M flooding unicast traffic

Ron,

Thanks for the the help, i changed
the mac-address age to 30minutes
and the flood never showed its ugly
face again.

Seems that 5min setting is either
not very good as the 4 routers
in question are constantly
passing traffic all day long
(peering point) so there is no way
in god's green earth any of the 4
mac-addresses could have staled out...
not every 3-4minutes

I contacted support and they said the
5min thing is default and wanted me to
check if any of the 4 mac-addresses were actually disappearing in the port status
fyi, the 4 mac's never removed themselves.

the dumb thing about this whole deal
is that all traffic passed as expected during
the flood, so basically the switch
was flooding away for its own benefit - definitely a bug.

Thanks a million for not callin me crazy :)


ciao
Romeo



Ardon
Trusted Contributor

Re: HP4000M flooding unicast traffic

Hi,

I am an HP Networking Engineer located in Europe but now in the US for two weeks and I happen to be working on this "issue".

In Europe we have heard about this Unicast Flooding as well 2 or 3 times. Although I do not know your exact network topology (and on which switch you saw the flooding) I can assure you that it is NOT a bug as the box is behaving as it should (although you can make it flood).
I should have my complete test report completed by Monday December 16th which I will submit to this forum for you and other people's reference.
If you have multiple switched environment you will probably not see the SAME flooding at all switches at a certain point in time because of the following.
Say you have a Switch A and B which are interconnected, both containing one client each (for convenience of explaining).
On Switch B you have also connected an additional client Sniffing.
Now Ping from the Client connected to Switch A to the Client connected to Switch B (NOT the sniffer client of course:-).
Your Sniffer Client should not capture those Pings (and I verified it does NOT).
Next step is to bring DOWN the Client on Switch B and see what happens with you Sniffer. Flooding, right? But why? Well that is pretty logical. At the moment you bring DOWN Client on Switch B that MAC will be flushed immediately from the MAC Table on Switch B but NOT(!) on Switch A (yet). What that means for the Ping is that it will NOT be flooded on Switch A but when it gets to Switch B where the MAC is flushed you WILL get flooding and thereby the Sniffer will capture it. Mind you that this Flooding will continue with this scenario as with every Ping issued the MAC timer on Switch A will be reset which is different from a test where you use a TCP application for example as that will try only a couple of times and then ceases to try (so not resetting the MAC timer as a result). After 300 sec. (Default) also Switch A will Flush the MAC of the Client connected to Switch B.
Most probably you will see Unicast Flooding in your environment only in just 1 direction (from X--->Y but not from Y---->X right?).

I will conduct the same test on a Cisco and/or 3Com Switch to show that this is common behaviour for your reference.

This is just a short explanation and, like promised before, a more extensive report will follow.

Talk to you later.
ProCurve Networking Engineer