topic Re: NFS server not responding in Operating System - Linux

NFS server not responding

Alex Lavrov. — Mon, 27 Jun 2005 02:29:51 GMT

Hello,

I have an nfs filesystem mounted on linux RHEL3 from HPUX.

Every 4-5 minutes I get in /var/log/messages this message:
kernel: nfs: server serv1 not responding, timed out

in "nfsstat -c" I see that:
calls retrans
708028 77304

It's not so good. What are the steps that I can take to solve this problem?

The network cards on both servers configured on 1000, Full, Autoneg ON.

No problems in pign either.

Thanx.

Re: NFS server not responding

Alex Lavrov. — Mon, 27 Jun 2005 02:39:52 GMT

Another thing, maybe it's related, I also get a lot of these messages:

kernel: application bug: dw.sapWT3_D00(3602) has SIGCHLD set to SIG_IGN but calls wait()

Re: NFS server not responding

Aris Cruz — Mon, 27 Jun 2005 10:28:29 GMT

Oh joy, nfs problems, they are always fun...

couple of things you can try.

1. change rsize=8192, wsize=8192 in mount options

2. RHEL3, i believe still has a default mount of udp, you can change the mount to tcp and nfsver=3

Re: NFS server not responding

Alex Lavrov. — Mon, 27 Jun 2005 11:24:49 GMT

Hello,

I'll try the first option.

About the second, it was TCP, so when the problems started we changed it to UDP it didn't help and it's already version 3.

thanx.

Re: NFS server not responding

Dave Falloon — Mon, 27 Jun 2005 11:37:00 GMT

watch out for soft mounts. The linux NFS faq ( http://nfs.sourceforge.net/ ) covers this under the heading at the bottom:

Why do I get NFS timeouts when I mount a Linux NFS server from my Solaris NFS client?

This applies to linux clients as well. These messages are shown when a client gives up trying to connect the the nfs server, this only happens with soft mounts, a hard mount has no timeout value. Two settings to mitigate this are "-o proto=tcp,hard" Since you are using Gig E you should also adjust the performance options.

--Dave

Re: NFS server not responding

Alex Lavrov. — Mon, 27 Jun 2005 12:25:17 GMT

Hmm, good point. I use soft mounts.

Before I assign points, I'll check it :)

Re: NFS server not responding

Dave Falloon — Mon, 27 Jun 2005 12:56:40 GMT

Please do as there could be other reasons for timeouts such as process limits on the server, sometimes you can run low on filehandles and start bouncing connections, etc.

--Dave

Re: NFS server not responding

rick jones — Tue, 28 Jun 2005 12:25:34 GMT

You might start looking at switch stats, ethtool stats (linux side) and lanadmin stats on the UX side. Also, netstat -s on linux and netstat -p udp and netstat -p tcp (depending on the flavor of the mount) on the UX side. Might toss-in a netstat -p ip if using UDP.

The idea is to see where packets are being lost and leading to the retransmitted NFS requests.

I don't have one for linux yet, but

ftp://ftp.cup.hp.com/dist/networking/briefs/annotated_netstat.txt

may be of some help.

Re: NFS server not responding

Steven E. Protter — Tue, 28 Jun 2005 14:17:09 GMT

Is the physical networking solid on this box?

I would suggest turning Autonegotiation to off and see if it gets any better. I would also see if other boxes have NFS problems with this box, perhaps even an HP-9000 or Solaris box.

The client and server nfs software should be up2date or yum current.

SEP

Re: NFS server not responding

Ivan Ferreira — Wed, 06 Jul 2005 13:46:49 GMT

We had connection problems with an ethernet adapter on linux, and we disabled the apic in the kernel and the problem was solved.

linux noapic

I also agree that the autonegotiation should be turned off, and configure fixed speed on both, the switch and the server.

Re: NFS server not responding

rick jones — Wed, 06 Jul 2005 13:50:09 GMT

In the original post, the speed was said to be "1000" which I would take to mean gigabit. In that case, autoneg _must_ be on - gigabit requires it.