ServiceGuard Linux - LVM Mirror

[Andreas Linnert]

Dear all,

we are using Service Guard (SuSE Linux Enterprise 10, SP1) for about one year now with a single storage system connected to both cluster nodes.
To increase availability, I have to set up host based mirroring via LVM to an other storage system now. Mirroring the data/application LUNs is no problem, but: How do I have to deal with the Quorum/Heartbeat/Lock LUN disk?

Thank you in advance
- Andreas

[Kapil Jha]

Quorum---there is no concept of 2 quorum it has to be one.
If its there I am not aware :)

HeartBeat: I do not know if there is something like high availability for heartbeat.
Only thing is to assign a dedicated LAN for heartbeat.

Lock LUN is same as quorum.

BR,
Kapil+

[Matti_Kurkela]

You can run heartbeat over multiple network interfaces in each node, and actually doing so is recommended.

If you can make your heartbeat connections reliable, the issue of quorum/lockLUN becomes less important. As long as each node has at least one way to hear others' heartbeat, the cluster can coordinate its actions without relying on cluster lock.

The cluster lock (whether a quorum server or a Lock LUN) is only really needed when the heartbeat connections are lost. Serviceguard will periodically check that the cluster lock is available, to ensure that it will be there when it really is needed.

The failure of a cluster lock *only* does not cause any interruption of Serviceguard services, but without a cluster lock, any further heartbeat link failures may not be recoverable.

The essential property of a cluster lock is that it *must not* be splittable: when one node possesses the lock, others must clearly detect that the lock is unavailable.

If the lock LUN is mirrored at the host level, it is possible that one node of the cluster can acquire the lock using one half of the mirror, while the other node can successfully get the lock using the other half. This *completely defeats* the protection the cluster lock is supposed to offer. This failure allows a split-brain situation, which causes filesystem corruption and data loss - a situation which Serviceguard is supposed to prevent at any cost.

You could have a "stand-by lock LUN" pre-configured to the storage system that does not contain the current lock LUN. That would be a LUN that is otherwise fully prepared to act as a lock LUN but not listed in the Serviceguard configuration as such.

If the active lock LUN was ever lost, you'd only need to enter one cmapplyconf command to make the system switch to the new lock LUN. This requires a version of Serviceguard that can handle lock LUN changes without stopping the cluster: the latest release A.11.19 can certainly do that.

Your configuration approaches that of an Extended Distance Cluster. For that, HP recommends using a quorum server (at a separate site) instead of a lock LUN. See this whitepaper:

http://docs.hp.com/en/10572/SGLinux.disasterproofdatacenter.pdf

MK

[Viktor Balogh]

>Quorum---there is no concept of 2 quorum it has to be one.

Yes, there's no sense to deal with 2 quorums at once. But if you're looking for a HA solution, the quorum service can itself be a ServiceGuard HA-package, (obviously it must be running on another cluster).

[Serviceguard for Linux · Accepted solution]

First, LVM mirroring is not supported by HP with SGLX. The only SW RAID method supported is the XDC product. As I understand it, LVM mirroring, especially with SLES10 SP1, is not supported with clustering. I don't remember the details.

If you wanted the most reliable quorum then you can use the Quorum Service in its own cluster.

Finally, as most are aware, since I have posted this previously, SGLX is being discontinued. See http://www.hp.com/go/sglxnews for more information.

[Andreas Linnert]

Hi,

first of all, thanks a lot for your replies.

Having two different quorum devices on two different cluster hosts doesn't make sense, of course. What we are trying to do is to cover a server failure as well as an error in the storage subsystem. Therefor, we had two alternatives: LVM Mirror (as we know it from ServiceGuard für HP-UX or HACMP for AIX) or a storage-based solution. The advantage of LVM would have been a continous availability although one of the storages fails.

Until I read the post that said "LVM mirroring is not supported with SGLX, the "standby lock lun" was a real good idea to me. But now, it seems we have to concentrate on "plan b". This may be a scripting solution or something manual.


- Andreas