Any way to stop nfs client from issueing file lock to nfs server

[Jackie Andrason_3]

Here is my scenario:

I have 2 HP rp4440 servers running HP-UX 11.11 and MC Serviceguard between them. Both servers currently nfs mount 2 filesystems from a Netapp NFS filer. The nfs mounts are not part of any cluster package--they are mounted at boot time via entry in /etc/fstab on both servers. There is only one package in the cluster and it starts up the oracle databases. There is a shared volume group with one filesystem that is enabled/mounted via the package. When we attempt a fail over test from the primary server to the alternate server (by killing power on the primary server), the alternate server starts up the package, but the oracle database startups fail because the Netapp filer shows that the primary server had nfs locks on the oracle control files (which exist on the filer) (dbas can move control files and copy backup copies and database will start up). Oplocks are turned off on the filer, so the filer shouldn't be keeping track of these locks, but they are being sent to the filer by the nfs lockd on the HP server. What we need to know (because netapp says it is on the nfs client side of things) is:
Is there a way to make the nfs locking local only?
Because the alternate will never need or use these nfs mounted files unless it is failed over, would/should/could we turn off nfs lockd on the 2 servers in the cluster?
Any other suggestions?

Thanks!

[Dave Olker]

Hi Jackie,

This feature is currently available on 11i v2 (11.23) at this time. To get this feature you would install patch PHNE_33099 (or any patch that supersedes PHNE_33099). The mount option is called "llock", so if you were to use this option the mount syntax would be:

# mount -o llock server:/local_fs /nfs_fs


Now, all that said, I still have to wonder why a file lock held by the NFS client would stop the server from failing a filesystem between nodes. HP's Highly Available NFS product has been able to do this for a couple of years (I know because I wrote it).

If the NetApp filers don't support file lock failover as part of the package migration, perhaps you'd want to try using HP-UX systems as your NFS servers? I know, shameless plug, but what the heck.

Let me know if you have any questions about the llock option or HA/NFS with file lock migration.

Regards,

Dave

[Sundar_7]

Jackie,

Try using the clear_locks command in the package control script to clear the NFS file locks. man clean_locks for more details

Sundar.

[TwoProc]

Well,

If the server that being "failed-out" is the problem, I'm sure you can try to kill lockd before you make the switch - providing you're not going to use the "failed-out" server any longer. Much like a reboot would clear it, I'd suggest doing a shutdown on all nfs level services on the "failed-out" server and then restarting it before taking the package back later on.

My discomfort with doing this is that I'm not sure this is so great for the database control files (I'm assuming they are locked for a reason) ... so lots of testing would be required ...

[Jackie Andrason_3]

Thanks for the suggestions, since I have OS 11.11 I can't install the patch, which sure would be nice. Any type of -nolock options or anything for 11.11?

The HPs are only NFS clients, not servers, and the NFS mount of the Netapp filesystems are not done with the HA product; they are just regular NFS mounts on both servers. Apparently when oracle opens the control files (I believe there is a local control file and a control file on the filer and both have to be able to be opened), the nfs lock daemon on the HP client sends something to the Netapp NFS server. We can run netapp commands and see the lock owned by the server that had the power pulled out of it. So, when the MC Serviceguard tries to start the database on the alternate server, the control files have errors because the lock owned by the primary server which is currently down. If I do a software failover, with a cmhaltnode or something of that nature, there is no problem, because it exits the database cleanly. We want to be able to not have the HP NFS client send the lock information to the Netapp NFS server when the file is opened, in case the primary crashes and the secondary needs to start the databases.

Does that make any sense?

Thanks a lot!

[Dave Olker · Accepted solution]

Hi Jackie,

Let me preface this post with the standard disclaimer:

This information is subject to change without notice. I'm giving you the best information I have at this time, but release schedules and patch contents can change prior to release.

Ok, disclaimer over.


I've been lobbying with the NFS lab to get this feature backported to 11.11 and I just learned that it will be released - or at least is currently scheduled to be released - in the next 11.11 NFS patch, which is currently scheduled for release in May 2006. The patch number is currently scheduled to be PHNE_34293. How's that for caveats?

If this is something you can't wait until May to get your hands on, I suggest you contact the Response Center and have them open a case with the WTEC organization and ask them for a pre-release (also known as GR0) version of patch PHNE_34293. WTEC/Lab is always looking for customers willing to test the patches prior to release and provide feedback. If you'd be willing to be a GR0 customer for this patch then contact the RC/WTEC and let them know.

Regards,

Dave

[Jackie Andrason_3]

Thank you Dave. We were kinda hoping the 11.23 patch would work on 11.11....but the good news is, I did put a case in with HP so I can ask for this option on that case. I gotta check with my management and see which way they wanna go, upgrade and patch with the 11.23 patch, or test the test patch. I wouldn't think the upgrade would be too involved. There will actually be several more systems with this same scenario, so we'll see. I will post results of whichever way we go in case anyone is interested.

I thank you very much for your time and responses.

[Dave Olker]

Hi Jackie,

If you actually have the possibility of upgrading to 11.23 you might want to consider that. There are other features of 11.23 that are currently not schedule to port back to 11.11 that you might find beneficial.

As an NFS client, one of the most important features is "forcible unmount". This allows you to forcibly unmount a hung NFS filesystem (caused by a down server, bad network, hung daemons, etc.) without having to reboot the client. This feature is a real life saver in production environments, like database environments.

Another feature available on 11.23 that won't be ported back to 11.11 is the "forcedirectio" mount option. This option is specifically designed for environments like yours, where the HP-UX system is an NFS client and a database server.

As I'm sure you're aware, most database programs like to manage their own cache resources, so in many cases the system buffer cache actually hurts performance. What the forcedirectio option does is tells the client to bypass the buffer cache for all access on that mount point. So if you're running a database on the HP-UX system and accessing the actual database files on the NetApp filer, using the forcedirectio option may give you a big performance boost.

Again, forcible unmount and forcedirectio are 11.23 features available right now. Coupled with the local locking feature, it would be an ideal operating system to run on a database server that accesses the database files via NFS.

Some things to ponder while making your decision...

Dave