dynamic route table entries

[Stephanie Nicholls]

On infrequent intervals (eg: once a month roughly) I am getting dynamic entries added to the route table of my production server. Refer attached route table with dynamic entries marked ***

I am trying to figure out what is causing this event to occur. From what I have read the card is not getting ICMP redirects as the flags on the route entries are just UGH not UGHD any ideas what it would be ?

Thanks

Steph

[sven verhaegen]

Hi

Well , how to start , there are only 2 ways of getting 'real' dynamic updates in the routing table , both will always end up with a D-flag , the first being a real ICMP redirect from a routing device the second being initiated by a routing protocol like RIP or OSPF , for this last case you should have GATED running on the system , check this out in /etc/rc.config.d/netconf

look for a parameter called GATED[0]=1 if it is there and you are sure you dont need anything like rip or ospf just put it to 0 and restart the system ... any other entry in the routing table that commes in there is for sure not 'Dynamic' in the real sence of the word it must be an actual manipulation of the system by either an application setting an entry in there (script ???) or a user creating a route , those 3 entries ar HOST routes visible by the UGH flags as would a normal icmp redirect be without the D flag , I would have a good look at any program that might run on a crontab that could execute a route add command .. by the way is it always the same IP's that are added ??? that could also point to a user adding things manually or a scritp being executed , is this thing happening with fixed intervals ?? that could point to a script executeing every (timeframe)

[Stephanie Nicholls]

Sven - thanks for the fast response.

No GATED is running on the box. /etc/rc.config.d/netconf displays GATED=0 and there is no other instance of it in my startup scripts.

The route entries are different every time, appear at irregular intervals and site history (the boss) tells me that this problem has been occurring for a number of years and occurred on the previous K class server they had here. The 2 times its happened since Ive been on site all the additional entries are within the same subnet but each time the subnet is different.

Only user connections are via Oracle Forms apart from sys admins and dba's, only crontabs are root and oracle, no 'route add' commands in any of it.

I have a really helpful networking guy, so if there's any pointers you can give me to get him to look for from the switch side that would be helpful too.

Steph

[sven verhaegen]

well steph ,

what is bothering me is the missing D flag , it clearly ppijnt to a non-dynamic entry , in allmy years of UX networking support I've never seen a dynamic route without a D-flag , I'm willing to believewe still have some sort of update mesage sent and the system 'forgetting the fla' but that would be really exceptional , besides the problem appeared on the other machine so probably it isn't system dependent but appliaction or external , the irregular nature makes it more difficult , I looked on hp-ux to see about deactivation ICMP redirect reaction but that isn't possible , a service request is opened for it but as ICMP redirect is integral and usefull function of hpo-ux it probably wont be altered , it is as far as i know possible to deactivate the sending icmp redirect on routers however , unforrtunately I don't know everything about cisco or other type routers so I just cant give commands or parametyers .. if I where the network admin , I'de start looking seeing if there is a correlation or delay before the problem reoccurs and try to capture packets sent to the server ona logging server filtering out ICMP type data and checking this log daat after a change was noted , perhaps we see a redirect anyway

[Bill Thorsteinson]

On my Linux system I run a program called icmpinfo that
logs ICMP messages to the syslog. I don't know if it
is available from the porting
center. If not you can get
the source from the Debian
site www.debian.org.

This should indicate if you are getting any redirects.

You may also want to wrap all
commands that can change the
routing tables with a script
that records the parameters.

[rick jones]

i concur that it would be highly unlikley that an ICMP-triggered route would not have a "D" flag in the output of netstat -rn.

is there anything special about the IP addresses? are they addresses belonging to any of your systems? what if anything runs on those systems?

the debian ICMP program may not compile out of the box on UX, but you could do something a bit cruder. download the latest version of libpcap and tcpdump from http://www.tcpdump.org/ and then run separate tcpdumps against your lan0 and lan1, with the tcpdump filter expression of "icmp" (I think that will show all ICMP traffic you receive). alternatively, you could turn-on nettl - i'm not sure if it can filter while collecting or only after.

another thing you could do is run a frequent cron/at job that deletes the routes that have "UGH" flags and then see what breaks :)