VLAN configuration in blade enclosure

[Barry Lowe]

Greetings. I'm trying to get a VLAN configuration working to enable two of my HP25p blade server to communicate directly via 192.168.X.X IP addresses. The switches that the blades are connected to are effectively black boxes to me and I've been told they've got a vlan id of 1 enabled and I should use that.

On both servers I have the following files, where Xs are replaced with numbers as appropriate:
ifcfg-eth2:
DEVICE=eth2
ONBOOT=yes

ifcfg-eth2.1
DEVICE=eth2.1
BOOTPROTO=static
HWADDR=XX:XX:XX:XX:XX:XX
TYPE=Ethernet
ONBOOT=yes
IPADDR=192.168.98.XXX
NETMASK=255.255.255.0
VLAN=yes

and have also run the command:
vconfig add eth2 1

When I run ifconfig the eth2.1 devices appear to be ok.

When I ping from one server to the other I always get 'Destination Host Unreachable', even if I explicitly choose the eth2.1 interface to ping from.

eth0 and eth1 are both also configured to 10.2.66.XX addresses with a default gateway of 10.2.64.1 on both.

Does this sounds like a problem with the switch or have I missed something from my linux config?

Thanks in advance.

[Steven E. Protter]

Shalom Barry,

No, so long as netmask is the same and the network portion of the ip address is the same you are good.

The "black box" needs to be looked at. The vlan configuration of the two ports is probably wrong.

Happens all the time.

SEP

[Connery]

Hi Barry,
I'm about 99% positive that this is an IP config problem on the server. A VLAN configuration problem on the switch will not produce a "destination host unreachable" error message on the server. If the VLAN config was wrong on the switch, you'd be getting "timeout" errors. "Destination host unreachable" messages almost always mean the IP stack on the server does not know how to reach the destination network because it doesn't have a route to get there.

So, here are a couple of things to try:
1. Make sure both servers are using the same IP network range and mask. Are both servers using 192.168.98.x or is one server using 192.168.98.x and another one is using 192.168.x.x? The information provided in your message indicates this might be the case.

2. What is the mask that you are using on eth0 and eth1 for IP address 10.2.66.xx? Since the gateway is 10.2.64.1, make sure you are not using 255.255.255.0. The mask should be at most 255.255.252.0.

3. Can the gateway on the 10.2.66.xx network get to the 192.168.xx.xx network? If not, trying removing the config for this IP address and gateway from eth0 and eth1 to see if it changes the behavior.

Good luck!
-sean

[Connery]

One more thing...
Unless the switch port is configured as a VLAN trunk port (using VLAN tagging), you should not be configuring VLAN interfaces on the NICs on your server.

If the ports on the switch that connect directly to the servers are only in VLAN 1 and are not tagged, then the server NICs should be setup as plain NICs (eth2) and not as VLAN enabled NICs (eth2.1).

If the ports on the switch that connect directly to the servers are in more than one VLAN, then VLAN tagging is enabled and you should use VLAN tagging on the server.

By the way, this should not cause a "destination unreachable" message, but you still need to make sure it's configured correctly.

[Barry Lowe]

Thanks for your responses.

I'm looking into the switches - they have vlan 1 configured by default, and vlan 200 for some other blades we are using. I'm hoping to give the interconnect a new vlan to use.

In response, all IP addresses and netmasks are set correctly. I have also tried with and without a gateway.

Incidentally I have also attempted a slightly different approach which was to have the ifcfg-eth2 and ifcfg2.1 files with the same contents and have seen different results from the config I mention in my initial post. I believe this is the difference between tagging all packets and only tagging packets specifically on vlan 1.

In the past I have not even considered vlans while I have been getting other aspects of the server configured and have not had problems with the networking - I presume this means that although tagging is enabled, the switch will allow packets through which are not tagged and treat them as vlan 1 by default?

I'm now pretty convinced I have a problem with the drivers or the hardware so have raised support calls with RedHat and HP.

[Barry Lowe]

Unfortunately it looks as though it's definitely a driver issue. I've been playing with eth0 and putting vlan 1 on and off.

Without any vlan, I am able to ping the neighbouring switch. Add vlan 1 and neither eth0 or eth0.1 can ping the switch. If I remove the eth0.1 config file and restart networking (so eth0.1 still exists) I am then able to ping the switch again. This is very strange behaviour so have raised a call with Red Hat and expect a solution sometime next month...