Lost packets?

[Anders_35]

Hmm.. this worries me:

We set up a script to ping various addresses, to check for delays and packet loss.

A 3424 is routing between two subnets, but they are on the same VLAN.
There is one 10Gbps link into the 3424. (see drawing).

All ping originate from a computer in subnet A. Pings to subnet B sometimes experience packet loss (and sometimes significant delays). But, pings to the router / gateway interface in subnet A also experience packet loss.
Pings to other subnet A addresses are always fine.

So.. this worries me. We have virtually no load on this network. Is the router fooling me somehow? Why would I experience ping-packet loss through that router, if it is not that busy?
For all I know, the 3424 isn't the problem, but it's definetly a candidate..

Anyone got any ideas?
I have no other indications of errors in the network.

Anders RM :)

[Matt Hobbs]

These types of issues are always difficult to troubleshoot.

1. Is the CPU usage normal on the 3400-24? If there is traffic that is destined to an unknown destination and there is no default route, this can cause the CPU to start rising. Ideally the CPU usage should be rarely rise above 15%.

2. Check the port counters on the devices involved. I've noticed on the 3400 that you may see dropped RX packets but I believe this is a cosmetic bug only so you can ignore that.

3. Is it possibly related to spanning-tree or Meshing in this case? Spanning-tree topology changes can cause problems (although you'd see this across the whole network, not just routed packets). Meshing can also be interesting as it will drop the first packet for an unknown source address to learn it.

Just for testing, maybe create a two new VLANs on the 3400-24 and connect two devices directly to it, and see how that goes.

[Anders_35]

Thanks for the feedback.

I don't think I have ever seen above 8% CPU during normal operation.
A few drops here and there, but they are so extremely rare, that I didn't give them a second look.


>3. Is it possibly related to spanning-tree or Meshing in this case?

Well, the mesh is always candidate, being the only part of this network that I really don't have any knowledge about. RSTP is not giving us grief, as far as I can tell, and with these ProCurves, we have had plenty experience with troubleshooting it :)

>"The mesh will drop the first packet"

That's interesting. But how often will there be a "first packet"?
I now have continuous ping-tests running, 4 second timeouts. The stream of packets never ends, so the adresses should not age out of any tables, or? Will the Mesh flush tables every once in a while?

[Matt Hobbs]

The 'first packet' would be if it had previously aged out of the mac-address table which is by default 300 seconds (5 mins). If you keep the ping running continuous then my understanding is that it should not age out.

[OLARU Dan]

You also might want to try a higher MAC aging time, for instance 1800 seconds.