NFS over TCP is not starting

[Don Mallory]

Hi there,

I have a couple of HP-UX 11iv1 (11.11) NFS servers, and both have exhibited a new "feature". Neither server seems to have NFC over tcp running anymore, even after one was rebooted.

server1 # rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  rpcbind
    100000    3   tcp    111  rpcbind
    100000    2   tcp    111  rpcbind
    100000    4   udp    111  rpcbind
    100000    3   udp    111  rpcbind
    100000    2   udp    111  rpcbind
    100024    1   tcp  53546  status
    100024    1   udp  56839  status
    100021    1   tcp  53547  nlockmgr
    100021    1   udp  56840  nlockmgr
    100021    3   tcp  53548  nlockmgr
    100021    3   udp  56841  nlockmgr
    100021    4   tcp  53549  nlockmgr
    100021    4   udp  56842  nlockmgr
    100020    1   udp   4045  llockmgr
    100020    1   tcp   4045  llockmgr
    100021    2   tcp  53550  nlockmgr
    100005    1   udp  57605  mountd
    100005    3   udp  57605  mountd
    100005    1   tcp  53578  mountd
    100005    3   tcp  53578  mountd
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs

It's quite clearly not there. It's not disabled, I've noted that in some messages a suggestion of NFS_TCP=1 should be in place, but I've never had this on either server.

nfskd -p returns:

server1 # /usr/sbin/nfskd -p
Getting nfskd kernel thread info
Error returned from kernel; errno = 14

I can mount things via UDP, but everything has gone a little off, and I'm reluctant to just start rebooting services/servers.

Thanks,
Don

[Dave Olker]

The "nfskd" command is not implemented so please don't use it.

I've seen this problem reported in the past.  Before we start collecting data, do you have the latest HP-UX 11.11 NFS patches installed?

Dave

[Don Mallory]

No, not the latest, but patches do not speak to why, after a reboot, a service that has had no issues for some time, all of a sudden doesn't start on boot.

Bundles:

  GOLDAPPS11i                   B.11.11.0712.475 Applications Patches for HP-UX 11i v1, December 2007
  GOLDBASE11i                   B.11.11.0712.475 Base Patches for HP-UX 11i v1, December 2007

Specific:

server1 # swlist -l product  | grep -i nfs
  NFS                   B.11.11        ONC/NFS; Network-File System,Information Services,Utilities
  PHKL_25238            1.0            11.00 NFS nfsd deadlock
  PHKL_25652            1.0            thread nostop for NFS, rlimit max value fix
  PHKL_28185            1.0            Tunable;vxportal;vx_maxlink;DMAPI NFS hang
  PHKL_29335            1.0            vx_nospace on NFS write.
  PHKL_34595            1.0            VM NFS umount fix
  PHNE_36168            1.0            ONC/NFS General Release/Performance Patch

[Dave Olker]

To be clear - is the problem:

A) When the system boots NFS/TCP is not enabled
B) When the system boots NFS/TCP works for some period of time and then suddenly NFS/TCP is no longer registered with rpcbind?

When this happens do you still see the "nfsktcpd" daemon running?

Dave

[Don Mallory · Accepted solution]

Hi Dave,

Thanks for the help. Yes, nfsktcpd is running. The issue is that at boot time, there is nothing disabling NFS over TCP, there is also nothing disabling NFS over UDP (I noted from other forum posts that this is an issue for some.)

With this in mind, tcp/2049 for NFS is not bound to RPC.

We ended up on the phone with HP 2nd level support, and after a series of diagnostic steps, determined that there was nothing wrong with the host. Restarting services did not fixe the issue. Oddly enough, a reboot did.

The rpcinfo -p now clearly shows:

    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs

Either way, the issue is resolved, but nobody seems to know why. What also doesn't make sense is that they host was rebooted as an earlier test, with no impact.

On the bright side, I learned a new command switch that I had not seen before. "nfsstat -m". Very cool

Best regards,

Don

[Dave Olker]

Hi Don,

Thanks for confirming that a reboot fixes the problem (at least temporarily) and the problem is one where the NFS/TCP service mysteriously un-registers itself for no apparent reason.  As I mentioned previously, this is a problem I've heard of many times in the past and I know the lab has released patches with fixes for this problem.  If you happen to encounter this again my advice would be to install the latest NFS patch for 11.11, which at this time is PHNE_41023, along with any dependent patches.  That way you will at least have all the known cases covered.

Regards,

Dave