NFS Problem

[Frantisek Merka_1]

Hello,
I have following problem, there was network failure and nfs between two servers is comming down, On one side (NFS server i have restart all nfs processes but, on client side there is no possibility to restart nfs.client or umount nfs filesystem ( there is a hard mount ), I have tried to kill rpc.statd, rpc.lockd and delete /var/statmon/state and /var/statmon/sm but without success.
Important is that this is production server and I cannot reboot.
Thanks for response
Best regards.
Frantisek.

[RAC_1]

You can tru this.

Run lsof(list open files tool) on nfs mount point on client. Kill all related process.

You should be fine after this.

Anil

[Mobeen_1]

Frantisek,
Have you tried to issue a force unmount on the client side. I would think that it might help as it unmounts irrespective of any open files. I would certainly give it a try

regards
Mobeen

[Shaikh Imran]

Hi,
Have you tried identifying the running processes by ps -aef and
kill those by fuser -ku ( carefully )

Regards,

[Frantisek Merka_1]

Hi, I didnot found force option for umount, on Solaris there it is "-f" on HP-UX I don`t know.
lsof i dont have instaled but a have used glance.
Any trial to kill some process that use NFS without success.
Frantisek

[Shaikh Imran]

Hi,

I know you have tried maximum possibilities.
Please try this also..and revert

#fuser -uck /
Where is the Stale filesystem.

Regards,

[Geoff Wild]

Sometimes, you can also manually mount the exported file system somewhere else - say /mnt.

Rgds...Geoff

[Frantisek Merka_1]

I know, thans very much for help, i cannot kill any process that use /usr/data/MIS, a have tried to stop, nfs.server nfs.core and nfs.client but without success. I think that my only possibility is to reboot server.

Thanks.
Best Regards.
Frantisek

[Frantisek Merka_1]

I have tried to mount the exported filesystem on other server too, and there was no problem. But this didnot work on my first server.

Regards
Frantisek

[Joe Short]

I have found the remedies in the attached HP document to be of great assistance in this situation. Hope it helps you too.

[Dave Olker]

Hi Frantisek,

There currently is no way to forcibly unmount a hung or stale NFS filesystem via the unmount command. However, we will be adding support for this feature soon in the upcoming 11i v2 update 2 (aka 11i v2 0409) release, where we will add a "forcible" option to the unmount command. This will allow you to forcibly unmount a hung or stale NFS filesystem without having to reboot the client.

In the mean time, I've written a white paper that discusses ways of getting hung or stale NFS filesystems to unmount. If your interested it's available at: http://docs.hp.com/hpux/onlinedocs/3929/ForciblyUnmountingNFSFilesystems.pdf.

Also, I would strongly advise against *ever* removing entries from the /var/statmon/sm directory. These entries are used by rpc.lockd/statd to keep track of which remote systems are performing file locks with the local system. Removing these entries can actually *cause* file lock stops responding to occur. If you are interested in the details let me know and I'd be happy to explain how.

Best of luck,

Dave

[Dave Olker]

One minor note, the URL I pasted doesn't work unless you strip off the trailing '.', so the real URL is:

http://docs.hp.com/hpux/onlinedocs/3929/ForciblyUnmountingNFSFilesystems.pdf

Sorry about that.

Dave

[Bill Hassell]

And just a note about NFS and production machines. Once you use NFS on a production machine, neither the server nor the network can ever fail or you will have big problems (as you have seen). NFS, while convenient to share files, is a real filesystem and may have open files, directories, inodes that are etting updated, etc. Take away the link and it is like ripping a disk drive out of the system--a lot of things will hang. You may want to evaluate the risks in using a network filesystem.