cluster config & name resolution

[Doug O'Leary]

Hey;

I had suspected this wasn't going to work and it's not. However, I'm now looking for some documentation which I can provide to the client telling them why it's not going to work.

I have three nodes each with two nics in them. The client insists that the IPs for both nics in each node be on the same subnet. So, my cmclnodelist looks like:

192.168.240.21 root
192.168.240.31 root
192.168.240.22 root
192.168.240.32 root
192.168.240.23 root
192.168.240.33 root

The nodes are using host based name resolution and are able to identify the proper node name with the proper IP address both forward and reverse ( host -> ip and ip -> host).

When I run the cmquerycl command, I'm getting "node ${node2} is refusing Serviceguard communication". I even set up .rhosts and verified those work.

One last bit of trivia: the client has hp-apa - autoport aggregation; however, I haven't configured it yet. My understanding is that will make both nics look like one so I will then only have one IP address for each node - and that definitely wont' work for serviceguard. Am I misunderstanding that?

Let me know what you all think and thanks for any pointers.

Doug O'Leary

[melvyn burnard]

>I have three nodes each with two nics in them. The client insists that the IPs for both nics in each node be on the same subnet.
This is not supported, they must be on separate subnets.

>When I run the cmquerycl command, I'm getting "node ${node2} is refusing Serviceguard communication". I even set up .rhosts and verified those work.
You should be using /etc/cmcluster/cmclnodelist and not .rhost, later version sofServiceguard do not use .rhosts, and even cmclnodelist is only used for the first creation/apply session
Take a read of:
http://docs.hp.com/en/5874/securingserviceguard0903.pdf
and maybe
http://docs.hp.com/en/6283/SGsecurityfiles0903.pdf


>the client has hp-apa - autoport aggregation; however, I haven't configured it yet. My understanding is that will make both nics look like one so I will then only have one IP address for each node - and that definitely wont' work for serviceguard.
Wrong, APa is supported with Serviceguard, documented in the manuals

[Doug O'Leary]

hey;

I'm a bit unclear on your answer... or maybe unclear on my question. I *used* the cmclnodelist and got the error listed above. The error message says check hostname resolution and, as indicated, that's working correctly.

I understand that APA works with serviceguard. My understanding is, for example, that you have four nics, you can use APA to team them together for two groups. Each group is going to have one IP address so that would satisfy the requirement to have two network connections.

This particular set up that they want has two nics total with two IPs on the same subnet. If I put APA on them, I'll have two nics in one group with one IP address.

Am I missing something?

Thanks for the reply.

Doug

[Duncan Edmonstone]

>> I'm a bit unclear on your answer... or maybe unclear on my question. I *used* the cmclnodelist and got the error listed above. The error message says check hostname resolution and, as indicated, that's working correctly.


I'm a little hazy on how Serviceguard uses hostnames/DNS in the context of checking security, but I can imagine a few situations where this "unsupported" configuration (yes Melvyn is entirely right there it is unsupported - your customer might want it, but they can't have it!) would cause this sort of problem - remember with two different NICs both with an IP in the same subnet, it's entirely possible for a request to be sent to one NIC, but be returned by the other - I can imagine this upsetting the nodename/IP/hostname checking algorithms...

Can we take a step back and establish _why_ your customer thinks he needs 2 IP addresses in the same subnet on different NICs? For most use cases there is generally a more sensible option that is actually supported...

HTH

Duncan

[Doug O'Leary]

Hey;

>>Can we take a step back and establish _why_ your customer thinks he needs 2 IP addresses in the same subnet on different NICs?


I'm probably damn near to the point of getting canned because I've been trying to get them to do just that. The IP addresses are only the tip of the iceberg. I've really been pushing back on them having direct access to all filesystems on all nodes of the cluster using VCFS on shared disk groups - to be accessed by straight Oracle - not Oracle RAC. Those two things, and the constantly moving deliverables target make this one of the more entertaining gigs I've had recently...


This client is migrating from a Tru64 cluster to MCSG. I can understand that they're comfortable in that environment and want what they're used to; however, tru64's cluster protected them whereas HPUX and MCSG won't.

I also have a suspicion that they want the package prebuilt, then migrate the application to it... "Cart, meet horse".

Anyway, thanks for the responses. Points forthcoming.

Doug O'Leary

[Duncan Edmonstone]

Doug,

I feel your pain...

I come across many places where the "this is how we did it, so that's how we'll continue doing it" mantra is in place... the thing is, different products and different OSs have different strengths and a good systems manager knows when to change to accomodate that. Serviceguard ain't like TruCluster - in many ways TruCluster had some superior features, but Serviceguard always had the advantage of being _simple_ and easy to understand/fix... non-standard implementations which try and make it look like TruCluster are a recipe for FAIL.

I once visited a site which had moved from UNIX to Windows - the old sysadmin there had rolled out cygwin to all the windows hosts and controlled the whole environment using shell scripts etc... it had worked well until he left (no doubt to get back to UNIX!), and when the company hired a new windows sysadmin to replace him, he was completely lost because of the totally non-standard implementation - we spent several painful weeks with me deciphering what all the scripts were doing and him re-implementing usual standard windows administration paradigms. My point is that trying to make things "the same" might make sense in the short term, but rarely works out in the end...

All I can suggest is some careful and thoughtful negotiation - point out that bespoke/complex solutions take longer for support to triage/fix and that they won't be able to pick up new sysadmins on the open market who can manage these sorts of things...

If that fails all I can suggest is that you lay it on the line with these guys (and their boss if necessary) - either you get a "supportable solution done the right way" or "a complex/unique/unsupported solution done their way".

Good luck,

Duncan