Testing / Verifying Auto Port Aggregation (APA)

[Navin Jain]

I am configured APA software (version B.11.11.07) running on HP-UX 11.11 with Dec 2004 patch bundle. The configuration seems to work ok. Here are the configuration details:

/etc/rc.config.d/hp_apaconf

HP_APA_INTERFACE_NAME[0]=lan900
HP_APA_LOAD_BALANCE_MODE[0]=LB_MAC
HP_APA_GROUP_CAPABILITY[0]=0
HP_APA_HOT_STANDBY[0]=off
HP_APA_MANUAL_LA[0]="0,2"

/etc/rc.config.d/hp_apaportconf

HP_APAPORT_INTERFACE_NAME[0]=lan0
HP_APAPORT_GROUP_CAPABILITY[0]=0
HP_APAPORT_CONFIG_MODE[0]=FEC_AUTO

HP_APAPORT_INTERFACE_NAME[1]=lan2
HP_APAPORT_GROUP_CAPABILITY[1]=0
HP_APAPORT_CONFIG_MODE[1]=FEC_AUTO

Output from lanscan -q
1
3
4
900 0 2
901
902
903
904

output form lanadmin -x -v 900
Link Aggregate PPA # : 900
Number of Ports : 2
Ports PPA : 0 2
Link Aggregation State : LINKAGG AUTO
Group Capability : 0
Load Balance Mode : MAC Address based (LB_MAC)

lanadmin -x -p 0 900

**** PORT NUMBER: 0 *******
Port FEC Mode : FEC_AUTO
Port State : UP
Port Group Capability : 0
Port Priority : 0

lanadmin -x -p 2 900
**** PORT NUMBER: 2 *******
Port FEC Mode : FEC_AUTO
Port State : UP
Port Group Capability : 0
Port Priority : 0

Question: How do I test if I am getting benefit of bigger pipe, ie load balancing.

1. I have tried to ftp a single file to a server and benchmark the timing. (time taken: 81 sec)
2. ftping the same file 10 times simultaneously to a server, does not show any benefit. (take taken: 750 sec)

Does any one know how to see the benefit of port aggregation ("larger bandwidth")? I can verify the lan failover feature and it is working.

Any suggestion or pointer would be appreciated.

Regards,

Navin Jain

[Bill Hassell]

You won't see any bandwidth improvement until the backbone is faster than your individual ports, or you can run multiple ftp tests from different machines. If all the ports are 100Mbits, you'll need several machines on the SAME switch to start transfers to the HP-UX box. Otherwise, you'll need a gigabit connection into the switch in order to bump up the transfer rates.

As far as failover, just randomly unplug a few of the links, then plug them in again while monitoring throughput.

[Steven E. Protter]

Shalom Navin,

You should see two benefits:

1) Your system should remain on the network with one cable or the other disconnected.

2) Increased speed if the core switch (backbone) is fast enough. Usually this is where performance expectations are not met.

SEP

[Arunvijai_4]

Hi Navin,

This guide should be useful to you,

http://docs.hp.com/en/J4240-90033/ch01.html
[HP Auto Port Aggregation (APA) Support Guide
Chapter 1. What Is HP Auto Port Aggregation? ]


-Arun

[Navin Jain]

Thanks to Bill Hassell and Steven Porter.

Here is my dilemma:

To test the bandwidth increase, I have benchmarked as follows:

1. Both ports on the source server set to 100FD
Target server port set to 100 FD.
No APA configured.
Time taken for ftping 10 files simultaneously: 77 sec

2. Both ports on the source server set to 10FD. Switch port at the same 10FD speed.
Target server port set to 100 FD.
APA configured in FEC_AUTO mode on the source server
Time taken for ftping 10 files simulteneously: 750 sec

which is roughly 10 times when using 100FD speed and 9.5 times of transfering
a single file on 10FD.

I would have expected at least 30% to 40% improvement in the network speed with 2 ports aggregation.

Regards,
Navin Jain

[Xianjie Zhang · Accepted solution]

Hi, Navin,
For FTP from a server to another server, the load balance algorithm must be changed to use LB_PORT to get a reasonable load distribution.
==
HP_APA_LOAD_BALANCE_MODE[0]=LB_PORT
==

In your case, since both ends have one unique MAC address, only one link will be used no matter how many links are available. Also, make sure that the switch's load balance algorithm is set to src-dst-port (Since you are using FEC, I guess it must be a Cisco switch right?)
Check the switch configuration manual to see if TCP/UDP port based load balance is available. (Some older ones or older firmware do not support this yet.)

Xianjie

[rick jones]

First - AutoPortAggregation - or for that matter any trunking software with sane packet scheduling algoirhtms - will _not_ improve the performance of a _individual_ "flow" It will only improve the performance of multiple flows - ie more than a single concurrent connection.

Now, with LB_MAC as the load balancing algorithm, I believe it is the destination MAC address that defines a "flow." This means that all traffic to a given MAC address will go over the same link in the aggregate. Similarly if you use the LB_IP (I'm guessing that is the name) algorithm.

If you want to have balancing across multiple links between a pair of systems, you have to use the load balancing algorithm that uses the TCP/UDP port numbers. I forget which one that is, but it would be in the docs on docs.hp.com

So, either have those 10 concurrent FTP's go to 10 different machines, or reconfigure the aggregate to use the more sutiable algorithm.

Finally, unless you _KNOW_ that autoneg does not work between your NICs and the switch, I would suggest leaving everything (NICs and switches) at auto. If you do hardcode the NICs to 100FD, make absolutely certain you have hardcoded the switch port(s) to 100FD as well. When/if you upgrade to Gigabit, autoneg is actually _required_...

[Navin Jain]

Hi Xianjie and Rick,

Thanks for your suggestions.

Changing LOAD_BALANCE_Mode from LB_MAC to LB_PORT increased the throughput. I have now tested 10 concurrent sessions going to 10 different servers and can see increased throughput.

Our network switch is Cisco 5000 series in the testing lab. Our network admin says that we have to use the â two adjacentâ ports on the network switch for trunking to work properly. Is it true with Cisco routers? Can I connect 2 NIC from servers to two different Cisco switches to create trunking and have some kind of switch redundancy?

Another question: Is there any option to see much traffic going through each NIC on the server (when using APA). Some thing similar to netstat â in, which shows number of packets In/out from each NIC.

Any document/paper which explains what does LB_MAC, LB_PORT, LB_IP means in layman terms?

Navin

[Bill Hassell]

APA's primary design goal is to provide redundancy with trunking as a benefit. But the term "Cisco routers" is ambiguous. There are many Cisco routers that do not work with APA. Naturally, Cisco uses different terminology (FEC/PAgP) when discussing trunking, load balancing and redundant links. Yes, there are some Cisco routers that know how to work together to provide redundant links between the routers. The routers must know about each other so they can backup each other in case of a failure.

[rick jones]

A device that is operating at "layer two" of the OSI model, either the traditional 7-layer model, or the updated 9-layer - see https://secure.isc.org/index.pl?/store/t-shirt/

is called a "switch" only if the device is operating at "layer three" (aka "network" or in this instance "IP" layer) is it called a router. OK, enough terminology nitpicking for now :)

Aggregating across more than one switch is possible with APA, but the last time I checked it only supported "active/standby" - unless and until support for "meshing" has been added to APA that is. As for Cisco kit, I've only ever played with ProCurve :) and eveven then I've not had to try to setup a trunk spanning two switches. I suspect the APA docs at docs.hp.com might discuss it in some detail.

[Navin Jain]

Thanks to everyone in resolving and helping in understanding the APA / network layers.

Closing the thread.

Navin Jain

[Dave Olker]

Hi Navin,

One last point before you close - if you're still interested in measuring the throughput of each individual port in the APA trunk you have a few options.

Glance can show each interface in the APA trunk, and you can use advisor mode scripting to display the inbound and outbound packet rates of each interface.

Also, the WTEC support group has a tool called "link-monitor" that they use to troubleshoot APA problems. It shows the real-time inbound and outbound traffic levels of each port in an APA trunk. I've used this tool on many occasions to determine if my APA trunk is balancing the outbound load well, if the switch I'm connected to is balancing the inbound packet load well, the total aggregate throughput of the trunk itself, etc.

Thought I'd pass that along in case you're interested.

Regards,

Dave