4 port trunk doesn't use all available bandwidth (2824 - 2848)

[Edmunde]

Hey guys,

Without getting into detail of the application, I am in need for 4 gig connection from switch to switch.

For some reason (which hopefully someone can explain) it seems that when I set up a 4 port trunk, my throughput through the trunk is only using 3/4 lines, and seems to be keeping the 4th for redundancy. If I unplug one of the active ports, the other jumps up and happily handles the traffic for me.

This is not what I want. I want all guns a'blazin.

I am able to get my 4 gig trunk if I team up 5 ports, (the 5th one now appears to be doing the redundancy), so I know it's not a throughput issue with the switches, they can handle that traffic.

I'm using a 2824 and a 2848, but I've also experienced the same problem with two 6108s.

Is there anyway for me to get FULL throughput from a 4 port trunk, without having to resort to a 5 port trunk?

In my lab, I'm using CAT6 for my trunk lines so I can get away with the 5 port solution, but in the real world, I will only have 4 fibres (and only 4 GBIC on the switches).

I've tried standard trunking, and LACP. Both show the same behavior (leaving at least 1 port of the trunk group to act as a fail-over, instead of just giving me all the bandwidth).

Firmware is up to date.

Please help. Any suggestions will be appreciated.

[Steven E. Protter]

Shalom,

No system ever works at the theoretical maximum. There is overhead in the NIC and the network infrastructure.

Have the switch port settings checked, this can play havoc with throughput.

lanadmin -x

the connections and see if they are all full duplex.

SEP

[Ketan Kotak]

Hi Edmmude,

This looks strange! So, need verbose text....

Start with some Qs:
1. Do you see all four ports in the trunk as active ones? Check using lanadmin -x -i

2. Which OS and APA version you are using?

3. What is LB_MODE (Load Balancing/Distribution Mode) set on the LA?

4. What type of traffic does the app generate? Do they have different src/dest TCP Port? or IP?

Now some APA internals:
* To have LB work for the load, you need to make sure that: (1) there are enough combinations of src/dest IP address for LB_IP. OR (2) enough combinations for src/dest TCP/UDP port number for LB_PORT.
* Actually, you choose LB based on which scheme gives you the maximum possible combination.
* Default LB is LB_MAC.
* Once one pair of src/dest is destined for one of the physical port in trunk, all packets of that combination will flow thru the same port - to assure ordering!
* You can reset the data flow distribution that APA is using for the trunk by using lanadmin -X -o command. APA will then start choosing the physical port for various combinations of src/dest depending on LB_MODE set. This may result is slight delay to stabilize the distribution - but is not significant in most cases.
* For more info, see APA DOC at: http://docs.hp.com/en/netcom.html#Auto%20Port%20Aggregation%20%28APA%29

Hope this helps.
Thanks and regards,
+ Ketan Kotak

[Edmunde]

Err...

Actually, it would appear as if I've posted this in the wrong forum, sorry guys.

But incase anyone was still interested. I'm using Microsoft (not my choice), and the application that I'm using to test the throughput is IOMeter.

I realize I wont get true 4 x Gb / sec, but when I use a 5 port trunk, I can saturate the 4/5 ports, giving me roughly 440 MB/sec. If I only use a 4 port trunk, then only 3 saturate, and I get about 310 MB/sec, or so.

I have 4 clients, each with their own gig NIC, blasting at a server with 4 gigabit NICs installed.

As mentioned, if I use a 5 port trunk, I get *nearly* full line speed from all hosts. If I go point to point, using cross over cables, I get the same speed (nearly 4 x 125MB/sec).

Just for the halibut, I tried created 4 individual VLANS, one for each host, between the switches, each VLAN having it's own connection between switches, and sure enough, it acted like a point to point connection and was able to obtain nearly full line speed from all 4 hosts.

As soon as I try to cram this traffic down a 4 port trunk, I only get 3 active lines, and one appears to be acting as a fail-over.


Sorry for posting this in the wrong forum, but I don't think it's OS related. If you can excuse my non-choice of an OS, and offer any insight to this issue, that'd be great.

Thanks for the help so far.

[Ketan Kotak]

Hi,

Next time be careful!

If you get >300MB for 4 port trunk, that means all four ARE in use.
It is the software's scalability issue! Either the trunking software or other part of OS may need to support nice scaling.

On HP-UX, APA scales very well upto 8 ports, with required patches.

So, you may want to consider HP-UX system to replace Windows!! :-)

Cheers..
+ Ketan

[Edmunde]

No. Only 3 out of 4 are in use.

I can monitor the traffic through the ports, and the 4th port is completely idle while the active 3 are saturated.

4 ports should give me a theoretical limit of 4 x 125 MB/sec. 500 MB/sec total. Considering the overhead, I get 440 MB/sec when going to point to point, OR, when I do the 4 seperate VLAN scenario that I describe above.

I'm absolutely certain that the 4th port in inactive (although enabled).

Anyways, again, sorry for posting in the wrong forum.

[rick jones]

We'll look for a post over in the switches forum(s).