Trouble virtual IP routing after Change of gateway server

[Roman Poon]

Hi,
i'm having trouble to configure my routing in an HA cluster. It was working for the last 6 months but we needed to replace the gateway server. Here is what happens.

We have a destination 172.18.0.10 and we need to send the virtual ip of the HA package. We did this by
route add host 172.18.0.10 172.16.15.1 4, 172.16.5.4 being the virtual address of lan1:1 and it was working perfectly. We passed the default gateway 172.16.10.10 onto the outgoing firewall 172.16.11.8.

It was working perfectly until we replaced the default gateway which is a complete different server.

NOW it takes a random route. In the example below you can see that traceroute involves 172.16.30.27 and not the default route 172.16.10.10.

127.0.0.1 127.0.0.1 UH 0 lo0 4136
172.16.16.2 172.16.16.2 UH 0 lan1 4136
172.16.15.4 172.16.15.4 UH 0 lan1:1 4136
172.18.0.10 172.16.15.4 UH 0 lan1:1 0
192.168.16.0 192.168.16.2 U 2 lan0 1500
192.168.20.0 192.168.20.12 U 2 lan3 1500
172.16.0.0 172.16.16.2 U 3 lan1 1500
172.16.0.0 172.16.15.4 U 3 lan1:1 1500
192.0.2.0 172.16.11.3 UG 0 lan1 0
192.168.159.0 172.16.10.10 UG 0 lan1 0
127.0.0.0 127.0.0.1 U 0 lo0 0
default 172.16.10.10 UG 0 lan1 0

lvk2:/root> traceroute 172.18.0.10
traceroute to 172.18.0.10 (172.18.0.10), 30 hops max, 40 byte packets
1 hvt301.pharmlog.de (172.16.30.27) 1.755 ms 1.603 ms 1.695 ms
2 burn.pharmlog.de (172.16.10.10) 0.169 ms 0.161 ms 0.139 ms
3 utgard1.pharmlog.de (172.16.11.8) 0.290 ms 0.296 ms 0.281 ms
4 utgard1.pharmlog.de (172.16.11.8) 0.376 ms 0.378 ms 0.362 ms


traceroute to 172.18.0.10 (172.18.0.10), 30 hops max, 40 byte packets
1 hvt301.pharmlog.de (172.16.30.27) 4.575 ms
2 vtl05.pharmlog.de (172.16.30.35) 3.252 ms 3.120 ms
2 * * *
3 * * *


I have found http://www11.itrc.hp.com/service/cki/docDisplay.do?docLocale=en&docId=emr_na-c01399683-1 to no avail, it still takes a random routing.

The HP-UX 11.11 + PHNE_35351 cumulative ARPA Transport patch or later for using
# route add net 172.18.0.0 172.16.10.10 1 source 172.16.15.4 are on the machine.

How can a replacement of the external gateway cause this behaviour and how can i act on the hpux?

[Steven E. Protter]

Shalom,

The gateway configuration is controlled not by the virutal IP but the master IP configuration in /etc/rc.config.d/netconf

The way to fix it, is a series of steps until resolution.

1) Reboot the server. Changes on the fly can cause this behavior.

2) Check the gateway itself for consistency.

3) Install the patch above if its not installed. This will probably require a reboot.

SEP

[Matti_Kurkela]

Welcome to the ITRC Forums, Roman!

If you replaced your gateway with a new machine, does the new gateway machine respond to pings? If its ping response is disabled (in the name of "security", usually), HP-UX will assume the gateway is inoperative and will stop using it.

If this Dead Gateway Detection is causing the problem, the solution is as follows:

1.) Disable Dead Gateway Detection until next reboot:

ndd -set /dev/ip ip_ire_gw_probe 0

2.) Delete and re-add the route setting for the problematic route, so the system "forgets" it has already detected it's dead.

3.) If the connection now works, you've confirmed the cause. Now edit /etc/rc.config.d/nddconf to disable DGD permanently. Add a group of settings like this: (replace "x" with the smallest number that is not yet used in the nddconf file... if there are no uncommented nddconf settings yet, begin with 0.)

TRANSPORT_NAME[x]=ip
NDD_NAME[x]=ip_ire_gw_probe
NDD_VALUE[x]=0

From this, the startup scripts will assemble a ndd command line like in step 1 and run it automatically at system reboot.

(Nitpick: you posted your question in a Linux section of the forum, but the HP-UX version information and your routing table output confirms you're asking about HP-UX. I'll ask the moderators to move this thread to the HP-UX section of the forum.)

[Roman Poon]

Hello,

1) the machine already has been rebooted, no change

2) The netconf is still correct; the gateway is running with the same ip-address

3) according to swlist -v | grep the patch is already installed.

Thanks for trying.
Greetings
Roman

[Roman Poon]

ok,
freshly booted

lvk2:/root> netstat -rni
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
lan3 1500 192.168.20.0 192.168.20.12 649 0 670 0 0
lan2* 1500 none none 0 0 0 0 0
lan1 1500 172.16.0.0 172.16.16.2 5461 0 4566 0 0
lan0 1500 192.168.16.0 192.168.16.2 643 0 669 0 0
lo0 4136 127.0.0.0 127.0.0.1 4301 0 4301 0 0
lan4* 1500 none none 0 0 0 0 0
lvk2:/root> netstat -rnv
Routing tables
Dest/Netmask Gateway Flags Refs Interface Pmtu
127.0.0.1/255.255.255.255 127.0.0.1 UH 0 lo0 4136
172.16.16.2/255.255.255.255 172.16.16.2 UH 0 lan1 4136
192.168.20.12/255.255.255.255 192.168.20.12 UH 0 lan3 4136
192.168.16.2/255.255.255.255 192.168.16.2 UH 0 lan0 4136
212.89.130.162/255.255.255.255 172.16.11.3 UGH 0 lan1 0
192.168.16.0/255.255.255.0 192.168.16.2 U 2 lan0 1500
192.168.20.0/255.255.255.0 192.168.20.12 U 2 lan3 1500
172.16.0.0/255.255.0.0 172.16.16.2 U 2 lan1 1500
192.168.159.0/255.255.255.0 172.16.10.10 UG 0 lan1 0
127.0.0.0/255.0.0.0 127.0.0.1 U 0 lo0 0
default/0.0.0.0 172.16.10.10 UG 0 lan1 0

adding virtual address

lvk2:/root> ifconfig lan1:1 172.16.15.4 netmask 255.255.255.0 up
lvk2:/root> netstat -rn
Routing tables
Destination Gateway Flags Refs Interface Pmtu
127.0.0.1 127.0.0.1 UH 0 lo0 4136
172.16.16.2 172.16.16.2 UH 0 lan1 4136
172.16.15.4 172.16.15.4 UH 0 lan1:1 4136
192.168.20.12 192.168.20.12 UH 0 lan3 4136
192.168.16.2 192.168.16.2 UH 0 lan0 4136
212.89.130.162 172.16.11.3 UGH 0 lan1 0
192.168.16.0 192.168.16.2 U 2 lan0 1500
192.168.20.0 192.168.20.12 U 2 lan3 1500
172.16.15.0 172.16.15.4 U 3 lan1:1 1500
172.16.0.0 172.16.16.2 U 3 lan1 1500
192.168.159.0 172.16.10.10 UG 0 lan1 0
127.0.0.0 127.0.0.1 U 0 lo0 0
default 172.16.10.10 UG 0 lan1 0

adding the destination

lvk2:/root> route add host 172.18.0.10 172.16.15.4 0
add host 172.18.0.10: gateway 172.16.15.4
lvk2:/root> netstat -rn
Routing tables
Destination Gateway Flags Refs Interface Pmtu
127.0.0.1 127.0.0.1 UH 0 lo0 4136
172.16.16.2 172.16.16.2 UH 0 lan1 4136
172.16.15.4 172.16.15.4 UH 0 lan1:1 4136
192.168.20.12 192.168.20.12 UH 0 lan3 4136
192.168.16.2 192.168.16.2 UH 0 lan0 4136
172.18.0.10 172.16.15.4 UH 0 lan1:1 0
212.89.130.162 172.16.11.3 UGH 0 lan1 0
192.168.16.0 192.168.16.2 U 2 lan0 1500
192.168.20.0 192.168.20.12 U 2 lan3 1500
172.16.15.0 172.16.15.4 U 3 lan1:1 1500
172.16.0.0 172.16.16.2 U 3 lan1 1500
192.168.159.0 172.16.10.10 UG 0 lan1 0
127.0.0.0 127.0.0.1 U 0 lo0 0
default 172.16.10.10 UG 0 lan1 0

checking the traceroute
lvk2:/root> traceroute 172.18.0.10
traceroute to 172.18.0.10 (172.18.0.10), 30 hops max, 40 byte packets
1 hvt301.pharmlog.de (172.16.30.27) 33.421 ms 1.475 ms 1.628 ms
2 burn.pharmlog.de (172.16.10.10) 0.153 ms 0.139 ms 0.122 ms
3 utgard1.pharmlog.de (172.16.11.8) 0.294 ms 0.279 ms 0.267 ms

since traceroute is blocked by the firewall (utgard1) it's ok to be restricted to utgard1, BUT why does it take (172.16.30.27) it's a switch...

routing on the firewall is enabled and the sftp is ok, BUT only if the switch has a routing to burn, even the virtual address is send correctly. If i use

route add host 172.18.0.10 172.16.10.10 1 source 172.16.15.4

lvk2:/root> traceroute 172.18.0.10
traceroute to 172.18.0.10 (172.18.0.10), 30 hops max, 40 byte packets
1 burn.pharmlog.de (172.16.10.10) 0.280 ms 0.132 ms 0.108 ms
2 utgard1.pharmlog.de (172.16.11.8) 1.230 ms 0.692 ms 0.194 ms

BUT
Jan 20 16:17:37 utgard1 kernel: DENY UDPIN=eth0 OUT=eth1 src=172.16.16.2 DST=172.18.0.10 LEN=40 TOS=0x00 PREC=0x00 TTL=1 ID=56081 PROTO=UDP SPT=37910 DPT=33439 LEN=20
Jan 20 16:17:37 utgard1 kernel: DENY UDPIN=eth0 OUT=eth1 src=172.16.16.2 DST=172.18.0.10 LEN=40 TOS=0x00 PREC=0x00 TTL=1 ID=56082 PROTO=UDP SPT=37910 DPT=33440 LEN=20

the source ip is not the virtuell ip (172.16.15.4)

How do i set the routing to
a) use the default gateway
b) to send the virtual-ip on outgoing data?