FC Adapter monitoring

[Joe Short]

I m setting up a cluster within a SAN environment. Each server (rp4440) has 2 FC HBAs installed. I am using HP-UX Mission Critical OS, and would like to configure the package to fail to the alternate node in the event that both FC adapters lose connection to the SAN. I have been trying a few different configurations, but without any success.
Can someone who's doen this offer some assistance?

Thanks.

[Jeff Schussele]

Hi Joe,

This is the main conundrum to "standard" HA scenario:

How to you guarantee the integrity of the data on the SAN if the links just drops suddenly? Can we be sure all transactions completed - successfully? Can we be sure that the SAN device is not just bogged down & the transaction will eventually complete - i.e. it's not just a temporary timeout issue?
How do we transfer control of the VGs to the other system if we can't "release" them from the primary system?

Bottom line with Service Guard is that this is problematic. The best you could do is to force the primary box to TOC & then spot that from the other node & start the packages up there semi-manually or heavily scripted because failover on disk access is not natively built-in to MC/SG. You need more robust systems & HA SW to do this - read *much* more $.

What you should & could do is make sure you have no SPOFs - i.e. multiple HBAs, multiple fibre switches, multiple FCAs on the array & use a good solid RAID in the array itself. Frankly, we've never lost all our paths to the SAN at the same time & the more critical the data, the more paths we use. We have one system with 8 HBAs - now we don't use eight switches - but we need the throughput that we can get with 8 paths, but we do use more than 2 switches & every HBA arrives at a different FCA port on the array.

Good Luck,
Jeff

[Joe Short]

That is the situation here. This is for a mission critical database. We have multiple HBA, switches, and even arrays. Ultimately this will be a campus cluster. So, if one side loses it's connection to it's array, the other should take over the package. I was under the impression that the EMS/HA monitor package could facilitate this scenario.

[Jeff Schussele]

Hi (again) Joe,

Ok - you're on the right track here with Extended Distance, Metro, or Continental Clusters.
See the following doc - Designing Disaster Tolerant HA Clusters:

http://www.docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/B7660-90014/B7660-90014_top.html&con=/hpux/onlinedocs/B7660-90014/00/00/8-con.html&toc=/hpux/onlinedocs/B7660-90014/00/00/8-toc.html&searchterms=cluster%7ccampus&queryid=20041004-134757

There are several ways to do it but they all use some sort of disk data replication or mirroring if you will. Some need a certain level of manual intervention & some don't. But the more sophisticated they get the more they cost.

HTH,
Jeff

[Jeff Schussele]

And here's several docs on EMS HA montiors - what they can do & how to install/configure & troubleshootthem:

http://www.docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/B5735-90001/B5735-90001_top.html&con=/hpux/onlinedocs/B5735-90001/00/00/6-con.html&toc=/hpux/onlinedocs/B5735-90001/00/00/6-toc.html&searchterms=HA%7cmonitoring%7cEMS&queryid=20041004-135906

http://www.docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/B5735-90001/B5735-90001_top.html&con=/hpux/onlinedocs/B5735-90001/00/00/8-con.html&toc=/hpux/onlinedocs/B5735-90001/00/00/8-toc.html&searchterms=HA%7cmonitoring%7cEMS&queryid=20041004-135906

http://www.docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/B5736-90046/B5736-90046_top.html&con=/hpux/onlinedocs/B5736-90046/00/00/23-con.html&toc=/hpux/onlinedocs/B5736-90046/00/00/23-toc.html&searchterms=HA%7cmonitoring%7cEMS&queryid=20041004-135906

HTH,
Jeff

[Joe Short]

Thanks, this is how i started this whole project. But I am unable to find any specific information on how to do it. This says what to do, but I need to get beyond this little obstacle. It's the final hurdle to completing this cluster.
Everything I've tried so far, just hangs the package in a halting state.

[Jeff Schussele]

OK - go to the Using HA Monitors guide - specifically the Monitoring Disk Resources chapter. Pay attention to the Rules & Parameters sections. You must not use the NO_TIMEOUT parameter. You must also naturally use the pv links...BUT...they *must* be in separate PVGs. Each PVG in a VG must have the same number of PVs. The package will depend on all the LVs in the VG if you wish to monitor via MC/SG.
There are further rules, but they're all spelled out there. You can even convert from "standard" LVM layouts to PVG layouts without recreating the VGs. But all the rules must be followed or you will have to recreate the VGs.

HTH,
Jeff

[Jeff Schussele]

OK - and finally - and maybe the most important. You have to make sure that the halt scripts are *not* dependent on disk I/O.
Because if you've lost both channels that command will just sleep and wait on I/O forever.
Just kill what you have to & hopefully it won't be dependent on I/O.
This may be the stickiest wicket.....

Rgds,
Jeff

[Joe Short]

I guess that about sums it up. In my specific configuration, this simply is not a possibility. I have my package that activates the VGs, and mount the file systems, before starting up the database. The run/halt script is mounting and unmounting the file systems, so it just hangs when there is a disconnect to the external disk.
I had hoped that there was a better way, but I guess not.
Thanks for the assist.