Load Balancing Network Traffic

[Mike Hassell]

I have two 100BT NICs running on a HP-UX 10.20 server. Currently I have each of these NICs attached to the same switch and subnet, each having a separate IP address (same mask as well).

This is a database server where connections are made via socket transfer to one of the NICs. I've been looking for a way to load balance the network traffic to this server and I've come across APA (needs 11.0?) and ndd (comes with 11.0, not on 10.20?), but haven't found anything on 10.20 that supports this type of scenario.

I would like for traffic to be received on one of the NICs and traffic to be sent out via the other NIC. Is this possible with HP-UX 10.20? I've looked into nettune and it seems to lack all the features of ndd, which limit my options greatly.

We're moving to 11.0 within the next 6 months, but I'd like to improve the network traffic before then :) Any ideas? Thanks.

- Mike

[melvyn burnard]

Currently there is no APA available for 10.20 , only a product known as LanMonitor J5080A, which has the failover functionality, but NOT the load balancing side and is only available for Servers, not Workstations.

[Tim Nelson]

FYI, something that I missed before.. APA does come with a price tag.. $1100 or so.

[Mike Hassell]

I realize that there is no APA solution for HP-UX 10.20, nor is ndd available. So what options do I have to balance the network traffic over these two NICs?

What did other SAs out there do with two 100BT NICs in their 10.20 servers in the past to overcome this obstacle?

The only options I have come up with is to configure the database (Sybase) to listen on both NICs and split up the client traffic via configuring half for one IP and the other half for the other IP. This of course would take some serious effort and I would like to do it from the OS side if possible.

I feel like I'm not taking advantage of the hardware that was in place when I came to this company and would like to utilize what they have to the fullest.

Should I setup the second NIC on a different subnet and use static routes to balance the outbound traffic? or is there *ANY* other solution for this scenario? Thanks.

- Mike

[A. Clay Stephenson]

Mike, your last posting caused me to do some really off the wall thinking about how to split the load between the two listeners on the same box. I assume that you have two
DNS entries e.g dbserverA and dbserverB. Currently all of your clients are connecting to dbserverA. Here's my hare-brained scheme worthy of Bugs Bunny. If your clients are using DHCP then you could and probably do have
multiple DHCP servers which of course dish out
IP addresses on a first-come/first-served basis.
If a client gets his address from DHCP server 1
then he gets a DNS server which maps dbserverA to one address and if he gets his address from DHCP server 2 then he gets a DNS server which resolves dbserverA to another IP address namely that of dbserverB.
I admit that this is unusual but you then do all of the work on the servers. When you upgrade to 11.x then all you have to do is make the changes in the DNS servers.

[Mike Hassell]

Clay,

I see what you are saying and it is a little abnormal, but I worry about a few things that might pop up with that sceneario.

1. If the DNS servers have the same hostname 'dbserver' for DNS1 (primary), and DNS2 (secondary) then I worry about creating ARP errors on the network with the same hostname referring back to two different MAC addresess.

2. The client configuration is a big issue here, because they have configured the clients using IP addresses (which is another issue all together) so the name change would only effect a small number of the clients that access 'dbserver'.

Right now I have it configured so the hostname maps only to the single NIC/IP Addr and the other one has no english name associated with it rendering it somewhat useless. It's a tough question and I've yet to find someone who has a solid answer on what my options are. Your thoughts are appericated. Looks like I need to upgrade to 11.0 sooner than I thought :) Thanks.

- Mike

[Mladen Despic]

Mike,

Not sure if this would work for you. It requires some work, but it's just a thought:

1. Configure 2 subdomains on your DNS server, say domain1.company.com and domain2.company.com .

2. On about half of your clients, use "domain1.company.com" to configure domain name and DNS search order. Use "domain2.company.com" for the other half. (Update DNS database accordingly.)

3. Assign host.domain1.company.com to the first NIC and host.domain2.company.com to the second NIC in DNS database.

4. Now, if the application's configuration accepts "host" entry (rather than host's ip-address) on the clients, you can configure
all clients with the same "host" entry. (This may work with Netscape Mail; not sure if it does with database clients.)

NOTE: Assuming the clients on domain1 and NIC1 are on one subnet, the clients on domain2 and NIC2 are on another subnet, this solution (if it works at all) may offload your routers as well.

Mladen

[rick jones]

if two nics on the same system are configured into the same subnet, they can each receive traffic, but outbound traffic will only get routed through one of them - I cannot recall if it is the first one, or the last one to be ifconfig'd.

so, you setup your DNS to point the "hostname" at the IP address of the interface on which you want inbound traffic, and then you manipulate the routing table (see netstat and route manpages) to make sure that the default route and the subnet route(s) point at the interface you want to use for outbound traffic.

now, i'd have to question why you would want to do that. about the only place where it _might_ make a difference in throughput would be with old HP-PB 100BT cards, when each card was on a _separate_ HP-PB bus.

[Mike Hassell]

Rick,

I see your point, but answer this for me. If there is no APA solution for HP-UX 10.20, nor the advanced capabilities of ndd, then why would a customer purchase dual nics, unless he/she wanted to use the HP server with routing capabilities?

Both of these cards are configured at 100 Full-duplex to the cisco switch, so you are correct about not really gaining anything by having one recieve and one send, since these NICs are HSC cards.

I guess the only way to really improve the network performance for this particular server would be to purchase a gigabit NIC. Then we're talking a little bit of money, since this is a K370 and they only make an HSC 1000Base-SX fiber card.

Looks like I'll have to wait until we've upgraded to 11.0 and then implement APA. Thanks for everyone's thoughts.

- Mike

[Mladen Despic]

Mike,

I think a typical multihomed system used to have each NIC configured for a different subnet. I don't know if two (active) NICs on the same subnet are recommended on 10.20, given what's available. But then again, if there is a good way to do it, and it works for you, why not?

Mladen

[rick jones]

Multiple NICs in the same IP subnet are not actively encouraged for 10.20 :) Multiple NICs in _different_ IP subnets are fine and dandy.

You can get some same-subnet load balancing if you are creative with your static routes, but it is a bit tedious.

Indeed, if you want to get 200 Mbit/s of outbound throughput, you need to bind the NICs together with something like APA, which as you have seen means gong to the only 40 months old OS versus the 80+ months old OS :)

Ndd and nettune are merely interfaces to the capabilities of their respective transport stacks. It is the 11.0 transport that provides the strong end systems functionality. Ndd is only there to enable/disable it.

I'm not sure how much of a boost you would see with an HSC GbE interface in a Kx70. I would check your CPU util today and guesstimate based on that. There is nothing fundamental to Gigabit Ethernet as a _technology_ that makes sending or receiving data any more efficient in the host CPU. In broad hadwaving terms, it takes just as many CPU cycles to send a packet over GbE as it does for FE, or even classic Ethernet.

Now, what you might get are more efficient _implementations_ - for instance, the HP GbE interface provides Internet Checksum Offload (CKO) which will make life easier on the CPU. The NIC can also be setup to reduce the number of interrupts per packet (at a possible latency cost)

[Servio Cabrera]

Rick jones: What about HyperFabric?, does it use a lot of cpu cycles like GE does?, What is your opinion about it?

[Mike Hassell]

Rick,

Good response, glad to see someone is on top of their networking stats. Actually, this same server that I've been referring to is being integrated with some linux servers to do some Sybase replication, which requires GbE. This will be a new private network that is segregated from the main network. So I've already purchased a HSC GbE NIC (A4924A), which takes up my last HSC slot in the K370.

It's due to be installed next week, so we'll see what kind of data throughput it can really do. The box is running with 4 CPUs and 1GB of memory, with CPU utilization as high as 90% at times, so I'm not sure if we'll really see a major performance gain, but management has decided to go the GbE route regardless.

Do you find using the default settings of the TCP/IP stack for 11.0 to be sufficient, or do alter the default config for performance gains? Just wondering. Thanks.

- Mike

[rick jones · Accepted solution]

birds and stones... :)

hyperfabric can help in two ways - frist by virtue of implementation - it has a _very_ efficient driver model and driver. that means that latency through the hyperfabric card will likely be much better than through the current GbE card. this can be examined by the curious with a netperf TCP_RR test using single-byte request/response pairs

second, by virtue of technology - it has an MTU of something just shy of 32KB. that means a bulk transfer will need fewer packets to transfer the data. fewer packets means fewer trips up and down the protocol stack, means less CPU consumption.

as for the all GbE private network, if the NICs allround support jumboframes, and the switch(es) as well you might enable that. it takes you to a 9000 byte MTU which will help with the bulk transfers and any time you send more than 1460 bytes at a time (the TCP MSS one gets with a 1500 byte MTU)