routing tables modified without knowing

[Anthony deRito]

Hello,

I have several UNIX servers running HP-UX 10.20 that were not able to communicate outside of our local network. After a little troubleshooting, I discovered that the routing tables contained the wrong gatways for several devices. Using netstat -nr, I immediatley noticed critical entries with the flags of UGHDM. The "M" stands for "modified : route has been modified from inital setting". (Usually this happens when the UNIX box receives an "ICMP redirect" from a router which causes the routing table entry to be re-defined.)

The problem was quickly resolved by removing the critical routes and adding them back with the correct gatways.

The gateway declared for the modified routes was the IP address of another CISCO router on our network! This router is only internal and knows nothing of the outside.

A clue here is that I know for a fact that I DID have problems with the true gatway.

But why did the 10.20 boxes just decide to plug in this other routers IP address as the gateway for these routing table entries?

I realize what had happened was normal but the question is WHY it happened. Why did the routing tables learn to use the IP address of this other router as the gatway once the system could not communicate with the only true interface that gets to the outside?

And yes I do know about the icmpinfo tool. And no lectures on static routes please.

Tony

[Berlene Herren]

The ICMP redirect error is sent by a router to the sender of an IP datagram when the datagram should have been sent to a different router. The only time you will see an ICMP redirect is when the host has a choice of routers to send the packet to. The "D" flag means the route was installed by an ICMP redirect.

A common use for redirects is to let a host with minimal routing knowledge built up a better routing table over time.

Berlene

[Anthony deRito]

The route entry was dynamically modified in the table. How did the system know to change the entry and assign it the wrong gateway? This is not an error but just how the routing tables work. Some device told the system to modify the entry. Was it the originating host or the router?

Tony

[Berlene Herren]

Tony, it comes from the router.

Berlene

[John Palmer]

Tony,

It is the router - I suspect that it is not configured correctly. I have seen this problem before but can't remember exactly what caused it, I'm no expert on Cisco's.

Regards,
John

[John Palmer]

Tony,

I have recalled some more of the problem that I saw...

The main WAN router (or its link) went down for a while and another router to a single network issued ICMP redirects for networks which it couldn't actually route to.

Hope this helps,
John

[Anthony deRito]

Hmmm... Interesting. Based on what I have heard from you guys and what I already know, here is my theory?

There were telecommunication failures on a T1 that the main router provided access to. Devices which required this gateway have detected undeliverable packets. The packets were automatically sent to the only other router available even though it did not go to the outside. At this point the routing tables were changed to reflect the new gateway. The new router sent ICMP events to the OS indicating that it was not possible to deliver the packets. OS decided to keep the routing tables incorrect instead of removing the bad gatways. I was left with a complete mess even after the T1 was repaired.

So far this is just a theory but I am still not satisfied. I need to understand the relationship between the routing tables and the routers and exactly what sequence of events led up to the change in the tables.

Thanks for your input.

Tony

[Sándor Szabó · Accepted solution]

Routes sometimes can become less than optimal. For example, it is possible for the router to be forced to resend a packet through the same interface on which it
was received. If this happens, the router sends an ICMP Redirect message to the packet's originator telling it that it is on a subnet directly connected to the router,
and that it must forward the packet to another system on the same subnet. It does so because the originating host presumably could have sent that packet to the next
hop without involving the router at all. The Redirect message instructs the sender to remove the router from the route and substitute a specified device representing a
more direct path.

You can disable ICMP Redirect Messages in the router, with "no ip redirects" command, or use this little awk script, scheduled every 15 minutes whit cron:

netstat -rn | awk '$3=/UGHD/ {system ("route delete "$1" "$2" ")}' > /dev/null

Best regards,
Sandor

[Anthony deRito]

This information is very usefull. It confirms what I have learned in the past few hours. The ICMP redirect messages can be quite anoying but I guess they serve a good purpose if the router that gets identified is a valid gateway. In this case it is not. I will confirm your command to disable the redirect messages and read up on it with Cisco. Thanks again for your time.

Tony

[Servais Alain]

The solution here is: avoid sending of icmp redirect messages by the router. In more complex situations, you might not be able to "search and destroy" or reconfigure the router.
How can hpux be configured to avoid its routing table is updated by hackers/routers/... ?