Problem network performance 5308Xl and 2848

[Alexander Samad]

Hi

I have a 5308XL trunk (LACP) to a 2848, with 2 x 1Gib cables.

On the 5308xl I have a Windows 2003 server with teamed (802.3ad Dynamic Fault Tolerance mode) - outbound round robin. - this has been configured up as trk2 - RDP

trk3 is LACP from the 5308XL to the 2848

on the 2848 I have trk'ed port 47-48 as LACP

I have 2 DL380's attached to the 2848 with 1G nics. They have RHEL3 (pdc1 & pdc2)

I am using iperf to run some network testing and what I have found is that the interconnect from the 2848 is only using 1 1G connection, I do see some traffic on the other connection and if I disable the active port it falls over to the second channel.

What this means though is that I can only get ~1Gbps max from these 2 boxes to the Windows box


testing I have done

Test1
from pdc1 to RDP ~1Gbps
Test2
from pdc2 to RDP ~1Gbps
Test3
from pdc1 to RDP
from pdc2 to RDP
Total ~1Gbps about 50% each

The interesting thing I have noticed is that on the rdp server all the traffic is coming in on 1 nic - not being balanced by between the nics.

I then ran the test in reverse mode - initiated from the pdc's but data running from rdp to the pdc's

I got roughly the same results 8(, with one exemption on the RDP server the load was balanced between the 2 nic's caused I have forced round robin, but still only getting a max of ~1Gbps.


So my questions
1 how do I utilise the full bandwidth
2 can the 2848 be setup to form a mesh connection to the 5308xl - I believe this might give better performance
3 How do I get the windows 2003 server to arp reply both the macs in round robin style or what do I need to do on the switches.

Thanks
Alex

[Shiraz Malik]

Hi Alex,

1. Procurve switches balance the traffic based on (SA,DA) entries table. when a conversation begins, the switches maps particular link in the trunk and the frames of that conversation uses the same link. It only changes the mapping when the mapped link goes down. So, the behavior you are observing is normal. You have configured the trunk correctly as you have tested it by disabling one link in the trunk. In case of communication in the opposite direction, it is not necessary for the switch to use the same link, because (SA,DA) entries would be different for that conversation.

2. Switch meshing is only possible with 5300, 3400, 6400 procurve switches so i won't go into the pros n cons because its not possible in your environment.

I hope this will help.

Regards,
Shiraz

[Alexander Samad]

Hi

I understand the SA/DA thing, this is why I did test 3 which is running to iperf tests at the same time so there would be 3 macs in play, I would have hoped that at some point in time I could use both the channels. I would presume the first test would use the first channel and then the second test would use the other


2
Well if there is a benefit in meshing - allows me to use the full bandwith then I might be able to upgrade the switch (2848).

[Alexander Samad]

Just tested it with 8 clients started up at the same time and they are all using the 1 channel, so I am not sure what is happening with the SA/DA load balancing. Seems like it is more active/passive setup !

[Shiraz Malik]

hi Alex,
Thanks for the feedback.

by default, the switch only considers the destination network address to decide about the link to be used. you need to run a command on your switch in global configuration mode i.e. "ip load-sharing by-host" then only it will consider the host address and not the network address.

I have also attached a document which contains some more details and clarification between load-balancing and trunking.

I hope this will help.

Regards,
Shiraz

[Les Ligetfalvy]

The 2848 does not support mesh. Also, beacause it has poorly allocated buffers, it suffers poor performance and needs to have the QOS tweaked. Read the release notes of the firmware for more details. Do not mix port speeds on the 2848 either.

[Alexander Samad]

Hi Thanks for the document.

I guess my miss understanding was I though it was based on mac's and not ip's.

But this riases the question that if it is on ip's shouldn't it be on source address and destination address and not just destination ? This is why I tried 8 nodes at the same time, although to the same ip address.

I tried to isse the command in the cli (whilst in config mode) and it doesn't exist.

So it seems like me having 2 channels is a waste except for redundancy, reason we have a very flat network in the lab 172.20/16 no routing 8) so it will never use the second channel.

[Shiraz Malik]

Hi Alex,

Trunk do use both channels but the load balancing is not adaptive and it does not calculate the path cost to choose the link to use for a particular conversation.

Meshing can certainly provide you a better mechanism for using both the link simultaneoulsy but it is only supported on 3400, 6400 and 5300. If you want to replace 2848 with a switch who supports meshing, than 3400cl-48G would be a good choice. Meshing actually has a decent mechanism to choose the link and load balance the traffic.


Regards,
Shiraz

[Alexander Samad]

Hi

Done some more testing on the 2848

basically 2 DL380G3 with RHEL3u4 with bonded nics connected to trunked ports on the 2848 I set round robin on the bonded interfaces

again running iperf and I get 500Mb, when I take it off the round robin i get back to the 1Gb speed, is this another similiar issue with the 2848 and the way it handles trunk's

NOTE - I don't have stp on