Load-Balancing Disparate NIC (speed) Cards

[Warren Landrum_1]

I've got a 7.3-2 OpenVMS cluster that I inherited consisting of 3 AlphaServer 4100s. Each system has 2 NIC cards in it - one 100MB and 1GB. The 100MB card is primarily used for intra-cluster (SCS) communications, as all 3 nodes are hooked up to a little 8-port Office Connect Switch via their 100MB interface, forming a sort of 'private Alpha LAN' if you will. The main body of users come into (telnet) the systems via the 1GB ethernet interface that is hooked up to a Cisco (the main) switch.

My question is this. One of our programmers just happened to notice that, on one of the systems, the receive and send packets are balanced almost 50-50 between the 100MB interface and the 1GB interface. Is there a way to weight these interfaces somehow so that most of the traffic comes in through the GB interface? I went into UCX> and looked at all the options available for SET INTERFACE and none of these seemed to be able to do the trick. Is this something that can even be done as far as TCP/IP is concerned?

Thanks in advance for your help.

warren

[Robert Brooks_1]

You can use SYS$SYSTEM:SCACP to limit the SCS
traffic to the 100Mbit interface. This may not be the most efficient way to "balance your load", however.

What utility was used to determine that the
send/receive packet counts were similar between the two ethernet interfaces?

Have you enabled jumbo frames on the 1Gbit interface? Does the cisco switch support jumbo frames? To enable them on the VMS side, you need to set bit six in the sysgen param LAN_FLAGS -- it's likely that the current value is 0, so to enable jumbo frames, set the value to 64 (decimal).

-- Rob (VMS Engineering)

[Tom O'Toole]

Rob,

Is setting jumbo frames recommended for SCS traffic? Thanks.

[Warren Landrum_1]

Rob,

I went into UCX as follows to determine the receive and send packet info:

$ UCX

TCPIP> sh int

and in the Packets Receive and send colums, itw was as follows:

Packets
Interface: Receive Send

WE0 4027117 54907082
WE1 3587472 5545651

Warren

[Warren Landrum_1]

"Have I enabled jumbo frames on the GB interface" - NO

"Does the Cisco switch support jumbo frames?" - WILL HAVE TO CHECK WITH OUR NETWORK MANAGER FOR THIS ANSWER

[Robert Brooks_1]

Tom asked . . .

Is setting jumbo frames recommended for SCS traffic? Thanks.

--------

Well, I guess it depends on your SCS traffic.

If you are serving disks via MSCP, then you may be moving large amounts of data over the NI SCS link. Lock manager requests tend not to be too big; I don't know if SCS is multiplexed (that is, several small messages bundled together in one SCS datagram).

As with most configuration- or site-specific issues, the best thing to do is to gather a bunch of statistics before and after enabling jumbo frames to see if anything good or bad happens!

As a matter of course, I enable jumbo frames whenever possible, but I'm not running a production environment.

-- Rob

[Robert Brooks_1]

Warren replied . . .


I went into UCX as follows to determine the receive and send packet info:

-----

That, of course, does not tell the whole story, as those numbers don't account for any SCS, DECnet, or LAT traffic (or any other protocol not part of the TCP/IP suite).

What problem are you attempting to solve?

It may be that your load is such that there is no need to do any load shifting from the 100Mbit interface to the 1Gbit interface.

-- Rob

[Warren Landrum_1]

Robert,

I think you asked te key question - 'What problem are you tryin gto solve?'

I don't believe there really IS a problem, The other 2 nodes in the cluster, from the same perspective, show most IP traffic going across the GB interface, and even on the node we are talking about, with the 50-50 distribution, I'm like "So what?". We are seeing no adverse effects on any processing, we are getting no complaints from users about lack of responsivenes, and it's probably been like this for years.

I'm only taking this programmer's (wanna-be-sysmanager) observation as an opportunity to post a question so that I could query my peers (and I admit I flatter myself by calling myself a peer with you guys, considering the quality and depth of some of the stuff I've seen out here!) to get a little more information and insight into something I had not really dealt with too much over the years.

With that being said, unless anyone has anything else to add, I'm going on vaction now (seriously) and when I get back in the office next Tuesday, I'll check out further replies and probably close this out.

Thanks both of you guys for looking at this for me!

Warren

[Robert Brooks_1]

I didn't mention this before, but I believe that having a separate switch and ethernet interface for SCS traffic is absolutely the right thing to do, on many levels, so with your configuration as it is now, you're certainly ahead of the game, independent of how the traffic is balanced across the interfaces.

What non-SCS traffic between the 4100's could be going over the 100Mbit interface?

-- Rob

[Cass Witkowski]

Be sure to check with Cisco whether the switch supports jumbo frames. We have a GBE switch and it doesn't. When we tried using jumbo frames the switches went out to lunch and had to be powered cycled.

Cass

[Warren Landrum_1]

got enough info for now from forum members to do further investigation