Switches, Hubs, and Modems
cancel
Showing results for 
Search instead for 
Did you mean: 

Understand Link Aggregation Throughput

foobarbaz
Occasional Contributor

Understand Link Aggregation Throughput

I am confused when trying to understand Link Aggregation (LAG) and if what I am trying to accomplish with LAG is possible. I have setup LAG between my linux box (4 NICS 1000FDx, SLES 10) and a SAN that I am running (4 NICS, iStoragePro iR16IS4ER) along with the switch sitting between all of this (ProCurve 5406zl). What I would like to accomplish is maximum throughput for my database (oracle data files) that will reside on my SAN. I have been doing some reading on LAG (specifically 802.3ad) and if I am understanding things correctly no single connection can utilize more than 1 single interface with LAG, at least in 802.3ad mode. Is this true? So for example the benchmark testing software I have been running aginst my SAN (ORION) seems to be utilizing only 1 interface out of the 4 interfaces I have when everything is configured for LAG. My benchmarks are proving this to be true and I cannot get more throughput on reads from my SAN than what a single 1000FDx port can give me even when I have LAG configured. In adition my benchmark software is telling me that I am maxing out at 3000 IOPS when doing small reads from my SAN when setup for LAG or running on a single NIC. I need at least 3x this amount according to the numbers I am pulling out of my currently running production database for my SAN to match current IOPS on existing hardware (local SCSI drives).

Thanks for any help.
2 REPLIES
Jaap Laaij
Frequent Advisor

Re: Understand Link Aggregation Throughput

Is your current config running over a single 1Gb Fdx?
Is the config of all the NICs in the LAG the same: Linux <> 2910al <> SAN box.
Jumbo etc.

The load-balancing within the LAG is normally done on a SA/DA basis (L3). L4 load-balancing is also possible but I don't think this will help you either.
foobarbaz
Occasional Contributor

Re: Understand Link Aggregation Throughput

@Jaap
Thanks for your input. Can you clarify your response in regards to whether or not LAG in 802.3ad mode with only a single connection can utilize more than a single slave out of the LAG even if the LAG has multiple slaves. From what I am reading this is not possible.

>>Is your current config running over a single 1Gb Fdx?
No I have 4 1Gb Fdx connections on each end (meaning the san has 4, the switch has 4 and the linux box has 4).

>>Is the config of all the NICs in the LAG the same...
As far as I can tell yes everything is configured the same on all ends. I could not get Jumbo frames to work for some reason though.

>>L4 load-balancing is also possible...
I believe you are referring to xmit_hash_policy (linux setting). And you are right the documentation I am reading says that a single connection will not span more than 1 slave.