Private network not being used

[Jakes Louw]

Hi all

Got a weird one here:
We have a server configured with several cards:- corporate LAN, private LAN, heartbeat1, heartbeat2.
For some years, we have been sending files across our Private LAN with no problems.
At some stage recently, we started getting problems, whereby RCPs/FTPs sent to a server on the Private LAN connect via the Corporate LAN.

Here's a nslookup:
Name Server: tcenh058.telkom.co.za
Address: 165.143.84.142

Trying DNS
Name: wblvh083s.telkom.co.za
Address: 192.168.2.115

As you can see, host WBLVH083S resolves to 192.168.2.115, which is the correct address on our private subnet.

However, here's a traceroute to the same server:

tcenh046:/)traceroute wblvh083s
traceroute: Warning: Multiple interfaces found; using 165.143.128.159 @ lan2 traceroute to wblvh083s.telkom.co.za (192.168.2.115), 30 hops max, 40 byte packe ts 1 WBLVH083.telkom.co.za (165.148.212.187) 38.286 ms 48.646 ms 24.671 ms

As you can see, the connection jumps out on the 165.148.212 subnet, and connects on the remote hosts' Corporate IP (165.148.212.187).

So being confused, I check out a ping:

tcenh046:/)ping -o 192.168.2.115 PING 192.168.2.115: 64 byte packets 64 bytes from 192.168.2.115: icmp_seq=0. time=54. ms 64 bytes from 192.168.2.115: icmp_seq=1. time=60. ms

----192.168.2.115 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss round-trip (ms) min/avg/max = 54/57/60 2 packets sent via:
192.168.2.115 - wblvh083s.telkom.co.za
192.168.2.16 - tcenh46s.telkom.co.za


So my logic says there is no way that this can happen.
The Cisco network guru here reckons I must enable static routing. But why have I never had this need before? And I can't recall being able to do this on HP-UX? Or am I being dense (old age)?

[Laurie Gellatly]

I'd like to see an netstat -nr from this host
and then I suspect you might need to add
a route to your host so that it knows which
is the right interface to use including the
hop count.
Which host interface is corp and which is private?
We had a similar issue just this week.

...Laurie :{)

[Jakes Louw]

Laurie

Here we go:

Routing tables
Destination Gateway Flags Refs Interface Pmtu
127.0.0.1 127.0.0.1 UH 0 lo0 4136
165.143.128.159 165.143.128.159 UH 0 lan2 4136
192.168.9.16 192.168.9.16 UH 0 lan4 4136
192.168.10.16 192.168.10.16 UH 0 lan7 4136
192.168.2.16 192.168.2.16 UH 0 lan6 4136
192.168.2.102 192.168.2.102 UH 0 lan6:1 4136
165.143.128.246 165.143.128.246 UH 0 lan2:1 4136
192.168.2.0 192.168.2.16 U 3 lan6 1500
192.168.2.0 192.168.2.102 U 3 lan6:1 1500
192.168.10.0 192.168.10.16 U 2 lan7 1500
192.168.9.0 192.168.9.16 U 2 lan4 1500
165.143.128.0 165.143.128.159 U 3 lan2 1500
165.143.128.0 165.143.128.246 U 3 lan2:1 1500
127.0.0.0 127.0.0.1 U 0 lo0 0
default 165.143.128.1 UG 0 lan2 0

[Steven E. Protter]

traceroute just does the best it can to trace the route based on the default gateway.

Obviously your overall setup seems to be making it think that private lan is somewhere out on the public Internet.

Ours works exactly the same way because we don't have a DNS server on the private network.

I believe if you set up a DNS server on the private network that only servers the private network, name resolution can be done.

Then you'll need to add a gateway on the 192 network in /etc/rc.config.d/netconf

We didn't want to go through any of that pain so all of our scripts use numeric ip addresses. That goes direct on the private lan and everything, including ignite boots and transfers works fine.

Your Cisco guy, is right. As to why things changed, I'd talk to the Cisco guy. They changed something that they did not think would effect you. It did. Perhaps your domain controller/DNS server is hooked to both networks or they have the networks connected by one of those neat little Cisco routers.

SEP

[Ron Kinner]

Curses, a lengthy reply just got eaten by the posting bug.

Try

nslookup wblvh083s

instead of

nslookup wblvh083s.telkom.co.za

or do a
traceroute wblvh083s.telkom.co.za

See if you see any difference.

If not look at your routing table. You have two IP addresses on the same subnet assigned to lan 6. Why? Is that new? Could be confusing the table.

Ron

[Brian Hackley]

Jakes,

You are using HP-UX 11.11 which has a different traceroute command than previously. It "guesses" which interface to use, and often gets the wrong one. So you should not be worried at all. Use -i to tell it which lan card to use. I look at your ping -o output -- it is telling you the truth!
Another option instead of -i is:
try -s option to specify the source IP address for traceroute to use. This often "works" when selecting the "wrong" interface happens. For example:
traceroute -s 192.168.2.16 192.168.2.115


Unless arp broadcast is disabled this command should return the MAC address from wblvh083s:
arp 192.168.2.115
Tip:
Use arp -an to list the entire arp table.

Tip #2:
Some customers copy over traceroute from an 11.0 system which works.

Tip #3:
You can tcpdump, Ethereal, or nettl/netfmt to check the packets to be certain as to where they are going.

Hope that helps,

-> Brian Hackley
p.s. Ron the lan6:1 looks like an MC/SG Package IP, or some other multiplexed IP address on the same subnet which is just fine.

[Jeff Schussele]

Hi,

I agree with Brian.
You've been bitten by the traceroute -i necessity.
I can't remember which patch changed traceroute, but now you *must* use the -i iface (or -s ip_addr) to insure you go out the proper NIC.
To me it appears it uses the first NIC output from a netstat -in command - but I'm not positive.

Rgds,
Jeff

[Sridhar Bhaskarla]

As far as I know, this is a feature on 11i that you will need to specify the lan interface with -i option when you do traceroute otherwise it will pick up the first found interface. So, don't believe traceroute.

Ping's output is correct.

Checkout the inPkt/OutPkt numbers on the interface holding your private network using 'netstat -in' and see if they are growing. That's a quick test.


-Sri

[Jakes Louw]

Hi all

SEP was the first to raise an alarm in my head: I know our network guys have been doing some background work on the various DNS servers, including moving the DNS master to a Windoze platform.

I'll just run this past the my network contact, and see what he thinks about any mismatch between DNS and the adjacent router's routing tables...

[Jakes Louw]

By the way, the problem was resolved, but I have no "fix": after checking the routing, we stopped the connections (in this case, huge RCPs), and restarted them WITHOUT any changes on the HP-UX side, and the connections were correctly made to the VPN.....
very suspicious, eh?