Performance on Procurve switches

[Tom Mazowski]

We have a network that consists of three 2848 switches and about ten 2650 switches. Most of the 2650's connecto to a 'Core' 2848 over copper and fiber Gbit ports. This Core switch has routing enabled for inter-vlan routing. The network has been in place for a little over 2 years, and we have recently noticed performance problems with data transfers and VoIP quality.

I have worked with HP phone support on a few occasions, but we have not been able to resolve the problems. The only indication of a probelm in the switch logs are the following:

1)Core HP switch shows missed packet buffers:
Packet - Total : 1998
Buffers Free : 1407
Lowest : 0
Missed : 4778
We enabled qos-passthrough, and this number has not gone up for the past 2 days.

2)The remote 2650 switches show a lot of RX Drops on the uplink to the Core 2848:
Drops Rx : 19,016,803
We had a lot of broadcast traffic coming from a Microsoft NLB cluster. This has been moved to a separate VLAN, but the RX drops still increment. There are no other errors on any of the 2650 ports or in the logs.

3)The ports that connect to our VoIP phone switches show the following deferred transmit packets:
Deferred Tx : 4218

High traffic does not seem to be an issue. Most of the gigabit links have very low utilization (between .5 - 2%)

The problems we are seeing are usually pretty minor and random. Recently there have been complaints with VoIP call quality. The logs on the VoIP system show that there is jitter and dropped packets. This is all LAN traffic.

There is also a test that I have doen tranferring a 650MB folder over the LAN. It starts off OK, but usually fails and I get a windows error saying "Network resource is no longer available." This has been done on different switch ports, different switches, and with different PCs.

Any suggestions on troubleshooting this would be greatly appreciated.

Thanks,

Tom M

[Pieter 't Hart]

I'd start with a test within the same switch / vlan / subnet.
do the copy between two systems here. This way you exclude the uplinks and routing as a cause of the problem. do this on all switches.

Do the vlan's cross switches, or are they contained within a single switch?
If possible try to monitor the cpu-utilization of the switches especially the core/router.

Do you use ACL's in your network?
Complex acl's contribute to cpu-usage even with low trafic.

[Tom Mazowski]

Thanks for the reply Pieter. We have tried that and it seems to be OK. The problems seem to be related to inter-switch traffic, even on the same subnet/VLAN. Whe I have looked at the CPU on the core switch, I have never seen it over 10%. We are not using ACLs.

After upgrading the firmware on the 2650's and rebooting, the file copy problem seemd to be resolved (for now). My main concern is the VoIP quality issues. The logs on the VoIP system indicate problems with traffic on the LAN, and the ports that connect to the IP phone switches are showing:
Deferred Tx : 4207

Do you have any idea what causes "deferred" transmit packets?

[RobB_8]

Interesting... not a solution, but a step closer... I have over 50 sites running various HP switches with VoIP at all of them and have no problems except one site, which has the same problem as you. This one site is a brand new site and the only site I have with 2848's. The configs on the 2848's are almost identical to my 100+ 2650's (which are fine). What kind of phone controller? Mitel?

[Tom Mazowski]

Rob,
We use a Shoretel phone system. Are you using the 2848 for VLAN routing? How about the other sites...do you use the 2650's for routing? Do you have any QoS configuratoin enabled?