nfs pkg and clients on same cluster

[Ra Jose]

I have 2 nodes SG cluster (11.16) - node1, node2. I have pkgA on node1, pkgB on node2. Need to create pkgC on this same cluster. This will be nfs pkg. pkgA and pkgB both read/write from/to the filesystems exported by pkgC. So, when pkgC is on node1, pkgA will have filesystems locally mounted but pkgB will have nfs mounted. When pkgC moves to node2, vice-versa is the case.
I have strange problem here when pkgC moves to node2, can pkgB still nfs mount filesystems exported by pkgC. Pls note that pkgB and pkgC both are on node2. pkgC exports filesystems and pkgB nfs mount these filesystems. Both on the same server. nfs client and server are running though. Please let me know your thoughts or if anybody has tried such configuration.

I have tried making a 2-node nfs cluster and making several clients nfs mount from this cluster. Here, both things are happenning on the same cluster.

Thank you all for your replies.
Rajos

[Stephen Doud]

Please investigate the HA-NFS toolkit product number B5140BA. The toolkit implements NFS in a highly-available package and allows implementation of cross-mounts.

The documentation for this product is located here:
http://docs.hp.com/en/ha.html#Highly%20Available%20NFS


Check http://h20293.www2.hp.com/portal/swdepot/displayProductsList.do?category=HA on how to purchase a license to use the product.

The license entitles you to a codeword that will unlock the product on the Application Software CD/DVD media.

[hpuxrox]

You will need to put a wait fuction in pkg B control script to verify that pkg C is up fully and exporting filesystems before starting pkg B on the failover node.

[Dave Olker · Accepted solution]

Hello Rajos,

You may actually find it preferable to have the local nodes use a loopback NFS mount when pkgC is on the same node. Yes, there will likely be a performance degradation going through a loopback NFS mount as opposed to directly talking to VxFS (some of this can be resolved with proper tuning), but there are times when a loopback NFS mount is preferable.

For example, if pkgC is running on node1 and node1 has processes and threads accessing the filesystem directly via VxFS, if you need to move pkgC to node2 you will likely need to shutdown any applications accessing pkgC before you can cleanly unmount the filesystems and move them to node2. However, if node1 were accessing pkgC via a loopback NFS mount, you would not have to stop the applications on node1 to unmount the underlying filesystem and move the package to node2. Once pkgC comes up on node2 the loopback NFS mount on node1 now becomes a real NFS mount (client and server on different systems) and your applications on node1 should not see any disruption except for the time it takes to migrate the package.

This is why HP added a new command-line option to AutoFS - "-L". This option causes AutoFS to not use LOFS mounts when the client and server are the same system and always use loopback NFS mounts in this case. We had a number of customers using ServiceGuard HA/NFS and complaining they couldn't migrate their NFS packages between nodes because one of the nodes had the filesystems mounted locally and were holding the filesystem busy when it tried to unmount. The solution - force the server to always use loopback NFS mounts in this case to allow the filesystem to be cleanly unmounted and migrated when needed.

I hope this information helps,

Dave

[Ra Jose]

Thank you Dave and others. The loopback nfs mounts helps me solve this issue. I am using nfs toolkit (have to see where I can get A.11.11.07) to implement my nfs pkg. I read HP docs and HP recommends not to use TCP ver nfs for this kind of configuration (loopback or cross-over mounts) due to problems clients have (getting hung) when nfs pkg fails over. Any ideas on this? The 11iv1 and 11iv2 both use nfs over tcp by default.
- Rajos

[Dave Olker]

Hi Rajos,

The manual is correct. There is an issue where an NFS/TCP connection doesn't immediately recover in the case where an NFS package migrates back and forth between cluster nodes. There can be a 5-minute delay, which is how long it takes the NFS client code to forcibly tear down the underlying TCP connection used by the NFS mount and establish a new one.

The recommendation is to use UDP to mount filesystems residing in the cluster.

Now, let me be absolutely clear about this - the UDP recommendation is only on the HA/NFS servers themselves - not on any remote NFS clients accessing these mount points. The only systems that are affected by this TCP connection semantic are the HA/NFS servers themselves (i.e. the nodes running ServiceGuard that can potentially mount and export the filesystems).

I always recommend customers use TCP for mounting filesystems from NFS servers unless there is a specific reason not to, and the same goes here. Any NFS clients, i.e. systems that are not members of the HA/NFS cluster, should use TCP to mount the filesystems from the cluster. ONLY the cluster members themselves should use UDP to mount the HA/NFS filesystems - either from themselves via a loopback mount if they own the package, or from another cluster member if that system currently owns the package.

I hope this is clear. I don't want you switching over to UDP throughout your entire environment. I only recommend UDP for the loopback mounts and cross-mounts within the cluster itself.

Regards,

Dave

[Ra Jose]

Thank you Dave and others for your valuable input. You are right that the remote nfs clinets can be on tcp over nfs and clients on SG cluster themselves can use udp over nfs. Thank you all again.
Rajos