Strange DNS issue

[Matt Hearn]

Hey all, we installed tcpdump on our database servers. When I run it and grep for the OLD DNS servers, and attempt to telnet into the box, I definitely see attempts to connect to the OLD DNS boxes on port 53.

I was able to reboot one of the development database servers, and now when I run tcpdump and telnet in, it makes NO attempt to connect to the old DNS boxes.

Can anyone suggest why this would be? I was under the assumption that /etc/resolv.conf was read each time a lookup was done, but it appears that it was read once on bootup and stored somewhere in memory!

Can anyone suggest a way to make the servers stop checking the old DNS servers WITHOUT rebooting? Taking an outage on the production servers would be ill-advised.

Thank you!!!

[harry d brown jr]

Each application is different, thus a database might "cache" DNS entries because it's faster that calling DNS. I guess the answer is shutdown and restart the databases, but better would be a reboot to catch anything else! What kind of Database are we talking about and what release level??


live free or die
harry d brown jr

[Matt Hearn]

It's LIMS over top of oracle, but that's not really the issue.

I'm wondering if there's something I can do at the OS level to make it stop looking at wpdns1,2 (old DNS boxes). It shouldn't, as they aren't in resolv.conf, but tcpdump shows that on incoming telnet sessions, they definitely are.

I've checked pretty much every other configuration file in /etc, and can't find any mention of wpdns1,2 or their IPs.

I've tried refreshing inetd, but it doesn't seem to make a difference.

[Todd Whitcher]

Unless the application is started from inetd refreshing it would not matter.

Could be possible that the application is using low level res_* calls. The reading of resolv.conf is done through res_init()

It definately sounds like the applications are cacheing the resolvers and not checking them each time as previously mentioned.

Stopping and starting inetd should have helped w/ inetd services like telnet.

[Matt Hearn]

Telnet is one of the problems, though.

When we turn off our OLD DNS servers, telnet no longer works.

[Todd Whitcher]

Interesting, so you have put the new DNS systems in /etc/resolv.conf, restarted inetd (inetd -k, inetd ) and the problem still exists w/ telnet ?

[Matt Hearn]

I hadn't done inetd -k; I figured inetd -c was good enough.

This might resolve the issue; I did a tcpdump, telnetted into the server, saw it go to wpdns1. Then I did "inetd -k" and "inetd" and repeated; it did not appear to go to wpdns1 that time.

We have a test window starting at 1pm, so I'll be able to shutdown the legacy dns systems and see what happens then.

Thanks for your help!

[rick jones]

IIRC all -c does is cause inetd to re-read its inetd.conf file.

[Rich Owens_2]

We had an issue where we replaced our HPUX DNS servers with Windows Active Directory. When we shut off the HPUX DNS servers everything worked fine for about 24 hours and then applications began failing.

We had to reboot all of the servers to get rid of the data that gethostbyname had cached about the old DNS servers.

[MarkSyder]

Matt,

Sorry to bump a two-year old thread, but it's been quite useful to me. I note you're still active on the forums, so perhaps you might like to give some points to the people who gave you (and me!) some very good answers on this thread.

Mark Syder (like the drink but spelt different)