- Community Home
- >
- Networking
- >
- Legacy
- >
- Switches, Hubs, Modems
- >
- Poor Gigabit Performance on 4108gl
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО08-23-2005 09:43 AM
тАО08-23-2005 09:43 AM
Poor Gigabit Performance on 4108gl
Card A - J4863A - 100/1000BaseTX
Card B - J4863A - 100/1000BaseTX
Card C - J4863A - 100/1000BaseTX
Card D - J4863A - 100/1000BaseTX
Card G - J4862B - 10/100BaseTX
Card H - J4862B - 10/100BaseTX
The gigabit modules are my focus here. Connected to the Gb modules are a few file servers and several desktop machines. The endpoints are running a combination of Win2k Pro, Win2K Server, Mac OS X, and Mac OS X Server. All of the desktop machines are running Mac OS X.
Users have been complaining for a while about sporadic poor performance. So, over the last couple weeks, I've thoroughly monitored the network from end to end. The servers and the client machines don't appear to be the cause of any bottlenecks so I've started focusing on the switch. My testing has consisted of mostly file copying from one device to another. This is a typical daily activity that users have noted as being slow. Throughout this testing I've been monitoring port counters and also have installed Procurve Manager.
For examples sake, I'll concentrate an modules A & B. Here's the port map:
A1 - Win2K Server 1
A2 - Win2K Server 2
A3 - Win2K Pro 1
A4 - Win2K Pro 2
A5 - Win2K Pro 3
A6 - Win2K Pro 4
B1 - Win2K Server 3
B2 - Win2K Server 4
B3 - OS X 1
B4 - OS X 2
B5 - Win2K Pro 5
B6 - Win2K Server 5
Major performance degredation is reproducible when 2 nodes send to a single node at the same time. For example if A5 & A3 are both copying a large file (1000MB for my testing) to A1, the transfer rate for each copy drops to about 8.5MB/s. Compare that to a copy that consists of just one port copying to another port. An A5 -> A1 copy transfers at about 35MB/s. Granted, the many-to-one scenario should be slower by its nature, however, I believe that it's excessively slow. Monitoring the port counters for each of the above copys, I see this:
A5,A3 -> A1: A5 has 1780 Drops Rx, A3 has 1807 Drops Rx, A1 has 0 Drops Rx
A5 -> A1: both ports have 0 Drops Rx
Note that the above example uses intra-module transfers. When doing inter-module transfers, I don't get the same results. For example:
A5,A6 -> B2: 0 Drops Rx on all ports, transfer speed is around 14MB/s
Note that there are no drops when doing an inter-module transfer. Even at 14MB/s, I believe that the switch (or something) is still limiting bandwith. When 2 machines are sending to the same port at the same time, and all are on Gb, I believe that the transfer speed should be greater than 14MB/s.
Now consider this...the Win2K machine on port A4 has an Intel PRO/1000 MT adapter in it. (The other Win2K machines all have Intel PRO/1000 XT adapters) Using Intel's ProSet config software, I can go in and tweak the NIC settings. Both the XT & MT adapters have identical configurable settings except one. The MT has a setting that's not available on the XT called "Interrupt Moderation Rate". I've found that this setting can make a big difference in both Drop Rx counts and transfer speed. For example if I do the same intra-module transfer as above, but use A4 as one the two sending ports, I get the following:
A5,A4 -> A1: 0 drops on all ports, both copys progress simultaneously at about 23MB/s
Even though only one of the sending devices has a configurable Interrupt Moderation Rate, that seems to be enough to speed things up. What I don't know then, is the Interrupt Moderation Rate a fixed value on the XT adapters and the Apple adapters? BTW, the OS X machines experience the same intra-module problems as noted above. In fact, they are worse in that simultaneous intra-module transfers to the same same port often result in one of the sending machines to completely drop the connection and fail the copy.
Or, is the switch responsible for causing the drops and slow transfer speeds? This switch was initially running SW v7.53 when I began monitoring and testing. I've since updated to v7.70. I've also implemented the qos-passthrough-mode feature (All tests above were done with this latest config). Through all of my monitoring wit PCM, none of the performance counters have reach a level anywhere near their warning threshold.
I'll talk to Intel too, but I wanted to post here and get any feedback from anyone who may have some indight into this.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО08-24-2005 01:30 AM
тАО08-24-2005 01:30 AM
Re: Poor Gigabit Performance on 4108gl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО08-24-2005 01:44 AM
тАО08-24-2005 01:44 AM
Re: Poor Gigabit Performance on 4108gl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО08-30-2005 12:21 AM
тАО08-30-2005 12:21 AM
Re: Poor Gigabit Performance on 4108gl
just two thoughts:
1) To test throughput, use throughput testing programs first. Testing file transfers depends on other factors, like disk performance and the quality of the protocols used (which in case of Windows SMB is especially adverse to any reliable testing). You should test with pure TCP streams that are sourced and sinked on the CPU. Have a look at NetIO, for instance.
2) When indeed interrupt mitigation seems to help you gaining throughput, this is a clear sign of the end systems beeing a limiting factor. This feature will do nothing but reduce the actual rate of NIC interrupts seen by the CPU to less than one per received frame. The switch is (almost) completely out of the equation here, except for maybe adverse effects of faster feedback from the host (which should not make a difference due to full duplex operation).
BTW, do you have 802.3 flow control active?
It could help prevent input queue overruns to some extend.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-07-2005 11:40 AM
тАО09-07-2005 11:40 AM
Re: Poor Gigabit Performance on 4108gl
My testing has basically involved two scenarios; 1) 2 Win2K clients simultaneously transfering to a Win2K server and 2) 2 OS X clients simultaneously transfering to an OS X server.
I've determined that the Win2K machines should use a TCP window size of 17520 bytes. With that setting, I get acceptable speeds and no drops reported on the switch. Above that and I get dismal speeds and tons of drops on the switch. This is likely related to this tech note: http://www.intel.com/support/network/sb/cs-000037.htm
By adjusting the registry keys
TcpWindowSize
GlobalMaxTcpWindowSize
I'm able to force the OS to use this desired value.
So the Win2k machines seem to be taken care of. The OSX machines however are suffering from a similar problem. Using iperf, which allows you to transfer data using specific TCP window sizes, I've determined that 17520 is also a good value to use for the OSX machines. iperf reports ~38MB/s when doing simultaneous transfers. Also, zero drops are reported on the switch. When using the OSX default of 64K, iperf reports ~6MB/s and I get >600 drops reported on one of the sending machines switch port. Similar to the way Win2K allows editing the registry to force a specific TCP window size, OSX has TCP parameters that can be altered as well (which work similarly to other UNIX OSes).
net.inet.tcp.sendspace
net.inet.tcp.recvspace
are the two params affecting TCP window size. I've tried changing them to 17520 however, simultaeous Finder copies still crawl along at 6MB/s and I get tons of drops reported on the switch.
Does anyone have in depth knowledge of these TCP params in a UNIX OS? Should't they take effect immediately after altering them? It's as if the OS is not using the new values and is sticking to a 64K window size.
Also, although I can clearly alter performance by making changes on the client & server, could this problem still be related to the way the switch operates, i.e. the lower TCP window sizes are just masking an issue with the switch.
In regards to droped frames on the switch, how can I go about determining what is causing those drops to occur?
Thanks for help on any of the above.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-07-2005 11:42 AM
тАО09-07-2005 11:42 AM
Re: Poor Gigabit Performance on 4108gl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-16-2005 11:01 PM
тАО11-16-2005 11:01 PM
Re: Poor Gigabit Performance on 4108gl
About a month ago I reported this issue to HP support and I too was asked to run the QOS pass-through fix. Worked for a little while, but now its back. Are your Wintel clients complaining about things slowing down? or is it just the OSX side of things?
Found another interesting thing yesterday. The uplink between our 4108 and a 5308 was being flagged by PCM as utilising alot of network badwidth. Checked config, both set to auto, upon running a network consistency check, the report stated that the 4108 end of the link was going at 1000Mb/s, and the 5308xl was running at 10Mb/s. Weird... Both running latest firmware. I've got an open case with HP and will call in the morning.
I'll let you know how I go.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-17-2005 01:18 AM
тАО11-17-2005 01:18 AM
Re: Poor Gigabit Performance on 4108gl
I discussed this issue on a couple forums with several people who know their stuff and the general consensus was that the switch was the culprit. One guy had a similar setup with a different Gb switch (a Dell switch I believe) and was not having this issue.
My next step is to try to get my hands on a different switch an run the same tests that I've been doing. So far I haven't had a chance to do that though.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-06-2006 08:58 PM
тАО03-06-2006 08:58 PM
Re: Poor Gigabit Performance on 4108gl
I've been told that with the 4108, the guts were essentially a different beast. In your case having the 10/100 cards and 10/100/1000 in the same switch do the following (if you hadn't already):
Do not plug any 10/100 devices into a 10/100/1000Mb card, it will interfere with all devices plugged into that card. plug all 10/100 devices into your J4862B - 10/100BaseTX cards and your 1000Mb clients only into your J4863A - 100/1000BaseTX cards.
In our case, we borrowed a 10/100 card and segregated the nodes in the prescribed way = better results.
J.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-07-2006 12:49 AM
тАО03-07-2006 12:49 AM
Re: Poor Gigabit Performance on 4108gl
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=680906