TX deferrals causing packet loss

[Tony Barrett_2]

At a remote part of our site, we have a 5406zl switch. This has a 10G uplink to the core (another 5406). Hanging of the rmeote 5406, we have a couple of other switches (2810, 2530 etc) that are connected via SFP fibre at 1Gb/FD and these have a few clients connected. During normal operations, everthing works fine, and there are no problems.

We have some backup servers that also connect to 5406 via copper (1Gb/FD), and at a couple of times during the day (mid-day) and after 8pm, these generate a lot of traffic while backups take place (generally 500-600Mb/s). Traffic is pulled across the 10G link, but it never goes above 7-8% utilization. The mid-day backups only run for 20 mins or so, the evening ones can run for hours.

When the backups start, some odd things happen. We start to see TX deferrals creeping up on the 5406 ports which the 2810 and 2530 are connected, which causes intermittent connectivity for connected clients. Our NMS shows packet loss to these switches at the same time. The uplink ports on the 2810 and 2530 show no errors at all, only the ports they connect to on the 5406. I have flow-control enabled on all uplink ports and spanning-tree enabled on edge ports , but there are no indications this is causing any issues. Once the high loading has finished, everthing returns to normal, and the deferrals stop.

All uplinks links are operating at full-duplex, which is strange as I thought deferrals only affected half-duplex links. I've checked all port settings, and they look fine, but I can't see why deferrals are incrementing under load on the 5406, unless it's a firmware bug (5406 is running K.15.16.0008.