hanfs package went down when DNS was not resolving

[vmatulis]

I received this message in syslog.
At the time DNS was not resolving, hanfs package went down. Any settings to check to prevent package being sensitive to DNS issues?

Oct 25 18:14:58 - WARNING: couldn't ping /usr/sbin/rpc.lockd. Retrying 1 more times before giving up

[Patrick Wallek]

Here is what I would do:

1) Change your /etc/nsswitch.conf file so it looks to files first and then DNS for hosts.

2) Put the host names and packages names for your cluster in /etc/hosts.

This way when DNS fails you are still able to look the names up locally.

[vmatulis]

I seem to have that already.

[Bill Hassell]

HA systems are critically dependent on networking and DNS represents a single point of failure. Often, the DNS server is not nearly as reliable as the HA systems and can destroy the reliability of these boxes.

The preferred method is to use /etc/hosts first, putting in the most important addresses and hostnames for the server. This does not have to be 1000 lines long since most servers only talk to a dozen or two systems. If the host is not found in /etc/hosts, then fallback to DNS. This has an added advantage: many commercial backup programs including HP's DataProtector will ask for the hostname for EVERY file during backup. This not only overloads the DNS server but creates a lot of network traffic. When hosts is used first, the hostname and IP addresses are in memory.

Change the file:

/etc/nsswitch.conf

to have the host: file read like this:

hosts: files [NOTFOUND=continue UNAVAIL=continue] dns

[Patrick Wallek]

Are you sure you are checking files first? If DNS going down effected NFS, then it could not resolve a hostname. This indicates to me that it is looking to DNS first.

Your nsswitch.conf should look like:

hosts: files [NOTFOUND=continue] dns

Another option would be to make your local machine a slave DNS server. That way if your primary goes down for some reason, you still have the slave.