Serviceguard for Linux NFS Toolkit

[Bryan King_2]

I am working with a customer who does not want to be the first kid on the block to implement any technology. They would prefer that others blaze the trail. They have asked me for any references of customers who have implemented the Serviceguard NFS Toolkit on either HP-UX or Linux. Right now, I'm recommending Linux to them. Any info you can share would be helpful!

Thanks,
Bryan

[Steven E. Protter]

It works well on both platforms.

I've used it on HP-9000 servers in HP labs and Compaq servers at HP-World.

SEP

[Florian Heigl (new acc)]

Tell them they're quite a few years late for being the first kid on the block regarding HANFS. Even if they tried clustering CodaFS on Linux they'd be 4 years late.

;)

[melvyn burnard]

Well the NFS Toolkit has been around and in use on HP-UX Serviceguard for at least 8 years.

As for the Linux version, it has also been a round a little while now, and has been implemented at many sites.

[Jean-Charles Ripault]

Hi,

I installed NFS on SG/LX twice at a customer site myself.

However, please be careful the toolkit is more bugged than the first RC of Win95 :-( You really want to re-edit it carefully before anything can work. Just be careful between others that you need to customize it if you want to use quotas as they otherwise generate a kernel lock that prevent unmounting the filesystem when you stop the package.

Also, you might want to kill the nfs daemon when stopping the package (even if you restart it), for the same reason.

When this toolkit has been created, it has been developped so several nfs packages could run simultaneously on a cluster. So there is a bizarre synchronization mechanism for the rmtab. You definitely want to rewrite it also.

To conclude, yes, it is possible to "clusterize" NFS, and yes, it has been done already (I did :-). But beware that it's definitely not piece of cake because of the very poor quality of the nfs protocol by itself (I tryied to stay polite).

Good luck.

[Informatikai Tanszekcso]

Could you provide some hints about these customizations? I am interested in using quotas on my NFS share.

Another question: "Managing HP ServiceGuard for Linux" writes that a cluster application should
bind to relocatable IP addresses. NFS toolkit specifies the address for the package, but I can't see how it is used with rpc.nfsd? I am planning to run multiple nfs packages and they may run on the same node. Are they able to bind only to the address of the package without interfere each other?

[Jean-Charles Ripault]

Could you provide some hints about these customizations? I am interested in using quotas on my NFS share.

JCR>> You need to explicitely run quotaoff before trying to unmount the filesystem, and redo a quotaon after mounting it, at least.

Another question: "Managing HP ServiceGuard for Linux" writes that a cluster application should bind to relocatable IP addresses. NFS toolkit specifies the address for the package, but I can't see how it is used with rpc.nfsd? I am planning to run multiple nfs packages and they may run on the same node. Are they able to bind only to the address of the package without interfere each other?

JCR>> That's not the way the NFS package will work, as per all the RPC packages. Whatever the setup you'll do, nfsd will always bind to all available IP addresses. That's partly why you use the "Pre-IP" setup. The way NFS has been setup in the nfs toolkit is that you will selectively export one or several filesystem(s) when you start and unexport them when you shutdown the package. However, there are still some cases where nfsd will keep a kernel lock on the partition that prevents you from unmounting it. My personal workaround is that if the fuser -k does not free the filesystem for unmounting, then I kill the nfs daemon, unmount the fs and restart nfsd. That way, I try to keep the disturbance to a bare minimum.

There, Hope it helps.

[Jean-Charles Ripault]

I forgot to mention that I completely re-wrote the rmtab synchronization process...

[Informatikai Tanszekcso]

What does Pre-IP setup exactly do?

I'd like to export two separate /home share for two different subnets (both having the name /home). According to that you write this is impossible? I wanted one nfsd to serve one of them, and an other to serve for another subnet (bindig their own IP).

sync_rmtab seems to be binary. How did you rewrite that? :-O

Thanks!

[Jean-Charles Ripault]

Pre-IP says that you start the daemon before you add the IP alias to the NIC.

I can't see how you'll do what you plan except by mounting a /home1 on /home in one subnet a /home2 on /home in the other if possible.

Finally, I removed the sync_rmtab and exported regularly the part of rmtab concerning one package in a file that I transfer to the other members of the cluster via scp (I do it as a separate service, which allows me to move the package in case this process falls). when I startup the package, I re-import this file in the curent rmtab.

This has been made necessary by the fact that no purge was never done within the rmtab with this process and that during the failover tests (10-20 failovers in a short timeframe), my rmtab ended over 20MB (it almost doubled each time) and the package took more than 2 minutes to startup at the end.