Multi-homed Cluster Networking issue

[Beazer]

I have an odd networking issue that I cannot seem to fix. I have two Es40's clustered together running 5.1A with the latest patch bundle. I have two network cards in each server the configuration is as follows:

Systems 1 Config
Nic1 - 10.10.10.55/255.255.0.0
Nic2 - 43.142.46.55/255.255.255.0

System 2 Config
Nic1 - 10.10.10.56/255.255.0.0
Nic2 - 43.142.46.56/255.255.255.0

I have one default gateway setup on each system as 43.142.46.3. I have not added any other routes to the systems.

I can ping anything on the 10.X.X.X side and I can ping anything on the 43.x.x.x side. I can FTP in and out from either subnet. However, I cannot TELNET out from either ES40. I can TELNET into the boxes from outside locations just not out.

When I try to connect to any remote system it just says "Trying to Connect" then eventually times out. When I do a netstat while it is trying to connect, it says that it is trying to use the 10.10.x.x. network versus trying to go out the default gateway on the 43.x.x.x side like all the other commands (ftp, ping etc). All other commands seem to work fine.
I tried to set up different routes but no luck.

I can telnet between cluster member by the way. Can anybody shed any light here or give some advice? Thanks in advance.

[Ivan Ferreira]

Can you post the output of your exact telnet command, that is, the host that you are trying to connect, and also, the output of netstat -nr.

[Beazer]

Here is output of netstat -nr
--------------------------------
stpes46> netstat -nr
Routing tables
Destination Gateway Flags Refs Use Interface

Route Tree for Protocol Family 26:

Route Tree for Protocol Family 2:
default 43.142.46.3 UGS 5 4618 ee0
10.10/16 10.10.10.55 U 1 1 bcm0
10.10.10.54 127.0.0.1 UH 0 1511 lo0
10.10.10.55 10.10.10.55 UHL 1 1163 bcm0
43.142.46/24 43.142.46.55 U 1 1 ee0
43.142.46.55 43.142.46.55 UHL 0 0 ee0
127/8 127.0.0.1 UR 0 0 lo0
127.0.0.1 127.0.0.1 UHL 10 24683 lo0
---------------------------------

When I do a telnet to IP 43.142.37.156 it just says "Trying 43.142.37.156......"

Here is output of netstat showing the connection in a "SYN_SENT" status

tcp 0 0 stpes45.jignet.com.2732 stpes41.ptg.co m.telnet SYN_SENT

[Al Licause]

If this is a cluster, what are you using for the cluster interconnect.....network or memory channel ?

What is the ip address of the cluster interconnect ?

Which interface represents the primary interface...i.e. the one that is associated with the hostname of the system ?

Do the other hosts you are trying to telnet to know the hostnames associated with your clusters....both hostnames and cluster aliases ?

Can you connect with any other application like ftp or ssh ?

Try enabling tcpdump and see where the packets are going and if they are receiving any type of response.

[Beazer]

Yes - this is a cluster and it is using a memory channel. The IP of the memory channel is 10.0.0.1 on on of the members and 10.0.0.2 on the other member.

The interface associated with the hostname is the 10.10.x.x interface.

The hostnames of all the systems involved including aliases are in the hosts files of each system.

Please note, that I CAN FTP out from this system just not TELNET. It appears to be trying to go out the interface associated with 10.10.X.X but FTP does not do this. I think this is were my problem is but I'm not sure how to get around it at this point.

I will have to setup tcpdump. It is not setup at this point. Thanks!

[Mark Poeschl_2]

That's a weird one. Trying out the tcpdump is good advice. Are you sure you don't have a firewall or some sort of security feature enabled in your networking H/W?

[Beazer]

Here is some more information:

On the 43.142.x.x side there is no firewall. On the 10.10.10.x side there is a firewall. However, the only default gateway that is setup is through the 43.142.x.x side. Also, to check this out I have created a rule on the 10.10.10.x side firewall to allow all traffic with no success. Remember, I can ping out and ftp out. It's like a routing issue where Telnet wants to go out the 10.10.x.x side and ping and FTP go out the 43.142.x.x side.

[Mark Poeschl_2]

Your netstat -rn output looks mighty odd. I don't see the cluster interconnect network in it at all. I believe even with 5.1A there was an 'ics0' pseudo-device associated with the cluster interconnect. Are you saying you've got three separate network connections:

43.142.46/24 on a 10/100 card
10.10/16 on a gig-E card
10.0.0.0 on memory channel?

What is the netmask on the memory channel?

[Al Licause · Accepted solution]

Take a look at each applications entry in /etc/clua_services.....look for the out_alias qualifier.

Your configuration is a bit strange in that you appear to be using the "secondary" interface or one that is not associated with the hostname for your main traffic pipe. To this end you have given it and not the 10.10. interface the default route.

I suspect you'll find each of the apps that don't work correctly to have out_alias applied to their entries in clua_services. As such they will use the address of the default cluster alias which is the one associated with the primary interface, in your case the 10.10.10 network.

I believe what you'll find is that when you telnet out, for example, it will use the alias associated with the 10 network and not the 43 network.

Remove the out_alias qualifiers from these entries, issue the cluamgr -f command and try again.

[Beazer]

Yaaaaahoooo! That was it! We have setup several of these here over the years but this is the first multi-homed cluster. We never had issues in the past. Thanks so much for everyones assistance!

[Beazer]

My problem was solved. The /etc/clua_services file needed modifed so Telnet would not use the default network path.