NFS stale file descriptor and close ()

[Bryan Strait]

Hi,

Quick question(s).

When NFS returns a stale file descriptor error during a call to close(), does it still release the file descriptor back to the OS? Or will it only release the file descriptor when the close actually succeeds? (Running HPUX 11v2 Sept 04)

We have a system in which hard mount points may occasionally lose connection to the NFS server whether or not all files have been closed. The application using these open files may call close() on the files after the connection is lost and the stale file handle error is returned. Eventually, when the mount point becomes available again, the application will reopen files on this mount point. I'd like to prevent any file descriptor leaks if possible. Can I assume that even if close returns with an error that the file descriptor is closed?

Is there a utility under HPUX that one can use to see the active file descriptors? If so, I should be able to find out if NFS closes the file descriptor or not in this case.

Thanks in advance for the help!

Bryan

[A. Clay Stephenson]

Bear in mind that one of the things that exit() does is close all file descriptors so the OS does this automatically when a process terminates normally or abnormally.
I point this out only to tell you that the system must handle this as a more or less routine event whether the files are local or not --- after all, that aspect of the file is invisinle to the application. In any event, from the perspective of the process there is nothing that you could do about it after the file descriptor is no longer valid.

[Steven E. Protter]

Shalom Bryan,

Its possible the file handle has been left open on the NFS server. It depends on whether the termination was abnormal or not.

I've recently fixed some issues with mysql server that stores its data on NFS, which is not a fully supported scenario, but that is where the disk space is.

The mysqld server crashes for unknown reason and won't start up again when the box comes back because as far as the nfs server goes the file was not closed properly.

In my case I had to re-run exportfs and restart the NFS service on the server which runs Linux. I'm not sure which action fixed the problem but now my mysql start script does both just to insure it comes up at boot time.

So its possible the file handle is still open and action will be required on the NFS server to fix it.

SEP

[Bryan Strait]

Thanks for quick response.

I'm thinking we may run into the same case you are having with your mysql server over NFS if the server thinks the file is still open.

I wonder if a CIFS/Samba server would do the same?

Bryan

[Bryan Strait]

The OS will definitely close and release the file descriptors when exit() is called, however, this application must continue running after close() has been called file from a disconnected mountpoint. Therefore, exit() is never called. Since the application may move on at a later point and open new file descriptors, I was concerned that close may never have release the file descriptor if it returned an error.

Thanks again for the great input.

[Frank de Vries]

A good utility is lsof

[Bryan Strait]

Thanks, Frank. I'll give it a look.

[Frank de Vries]

Hi,
Brian

How did it go with your NFS and lsof,
was the advice it worth a few points ?

Thx

[Dave Olker]

Hi Bryan,

I'm not sure if you're checking this thread any more, but I wanted to reply anyway in case others happen across this thread.

One point I'd like to make is that the NFS server does not really "open" files or keep file handles opened while clients are accessing them. The NFS server code runs in the kernel and uses device/block interfaces to directly access either the raw blocks on disk or any pages in cache. It doesn't keep references to these files "open" in the traditional sense.

If there is a file descriptor leak involved with NFS it would have to be on the client. If there is a file descriptor leak occurring on an NFS server then it would be caused by an application running on the NFS server, not the nfsd/nfsktcpd daemons/threads themselves.

As for your original query about stale file descriptors, are you talking about getting an actual ESTALE error or are you talking about a "stale" mount point where the server is unavailable? I'm trying to understand if you think you're getting an ESTALE error when you shouldn't be. However, if you're just talking about a hung filesystem caused by a down server then that's fine.

Regards,

Dave

[Bryan Strait]

Please forgive me for the late replies. I tried the lsof. It does indicate that the OS actually closes the file descriptors. I can see the file open before the close using lsof. After the close, it indicates that the file has actually been closed.

Dave,

Thanks for the background on the NFS file server-side. It was actually a directory timing out; not an ESTALE.

Thanks again for the help from everyone.