Timing test suggests that connection to lan0 is going through lan1

[PaulHoffrichter]

I did four timing tests with some results that I do not understand.

Have HP-UX 11i v3 (11.31) on rx2600.

A simple test server program resides on rx2600, and a simple client test program on a VM Linux platform.

lan0 (19.140.131.32) is intl100 NIC (100 Mbps)

lan1 (19.140.131.67) is igelan NIC (1 Gbps)

1. client connects to lan1 and sends 1GByte of data to server - expected high data rate; got 650 Mbps (but lower than test2)

2. client connects to lan1 and receives 1GByte of data from server - expected high data rate; got 921 Mbps - Good

3. client connects to lan0 and sends 1GByte of data to server - expected low data rate; got 94 Mbps - Good

4. client connects to lan0 and receives 1GByte of data from server - expected low data rate; got 930 Mbps - Surprised

Why does test 1 consistently run at a lower rate than test 2?

Why does test 4 look like the client is actually connected to lan1?

Not sure if the following helps, but here are the routing tables for the rx2600 server:

# netstat -rnv
Routing tables
Dest/Netmask                    Gateway            Flags Refs Interface  Pmtu
127.0.0.1/255.255.255.255       127.0.0.1          UH    0    lo0       32808
19.140.131.67/255.255.255.255   19.140.131.67      UH    0    lan1      32808
19.140.131.32/255.255.255.255   19.140.131.32      UH    0    lan0      32808
19.140.128.0/255.255.252.0      19.140.131.67      U     2    lan1       1500
19.140.128.0/255.255.252.0      19.140.131.32      U     2    lan0       1500
127.0.0.0/255.0.0.0             127.0.0.1          U     0    lo0       32808
default/0.0.0.0                 19.140.128.1       UG    0    lan1       1500

[Duncan Edmonstone · Accepted solution]

What you are seeing is entirely expected when you have two seperate physical NICs configured with the same IP subnet. There is very limited support for the config you have, and you need to understand the implications of it. If you have a support contract you should ask HP support to show you support document emr_na-c01031662-4 which gives you the formal support statement on this.

The rules in the routing table are applied from the top down, so in your case anything going to the 19.140.128.0 subnet, or anything beyond the default gateway which is on the 19.140.128.0 subnet will be sent out of lan1. That is correct behaviour.

If you want IP responses to leave out the interface they were received on you need to look at the ndd parameter ip_strong_es_model - but I would advise against that unless you _really_ know what you are doing.

Can I ask why you have configured the 2 seperate NICs in the same IP subnet?

[Kris_Knigga]

To increase receive speed on your 1Gbps NIC, you may look into tweaking the tcp_recv_hiwater_def setting via ndd.  I'm guessing it might be too small (especially if you have kind of high ping times between the two machines).

HP-UX 11i TCP/IP Performance White Paper

Also, if you are writing this data to disk, you might be running into a disk bottleneck.

[PaulHoffrichter]

@Duncan Edmonstone
Thank you for your reply. I appreciate them.

>> why you have configured the 2 seperate NICs in the same IP subnet?
The platform is used as a workstation for testing and research purposes. The network admin set the platform up using DHCP. I am sending a note to the new network admin letting him know of the potential problems with this setup, and will point out the document emr_na-c01031662-4.

After reading emr_na-c01031662-4, I think I understand those test results. In Test 3 the client connects to lan0, and since the server has inbound data, then lan0 is used; hence the expected lower data rate. In Test 4, the server has outbound data, and since the last ifconfig was for lan1, then lan1 is used; hence the higher data rate.

In some of the tests, I added tcpdump to the mix. In some tests, I got unusual results in the collection. But since HP does not officially support this lan configuration, I will see if we can change it to a true multi-homed one (i.e., different subnets), and then use tcpdump again.

@mkdlxk
I was primarily just trying to understand why the client connection to lan0 was going through lan1 in one of the tests. As the tests show, most of the speeds are are .7-1Gbps. Note that the data rates shown are for the body of the message, and does not include the tcp/ip/ethernet overhead; and this data does not involve disks. I have saved the referenced document for future reference. Thanks.

Regard,
Paul