Networking and Service Guard

[Alexis_2]

All,

I'm creating a 2 node cluster in my lab. The package comes up perfectly on the primary node but, when I try to fail my package over to the second node, I get the the following error:

Node "neldbs3": Adding IP address 192.168.118.51 to subnet 192.168.118.0
cmmodnet : Unable to verify IP address 192.168.118.51.
cmmodnet : 192.168.118.51 might already be configured as a heartbeat IP address.
ERROR: Function add_ip_address
ERROR: Failed to add IP address to subnet
Jun 10 23:48:33 - Node "neldbs3": Remove IP address 192.168.118.51 from subnet 192.168.118.0
cmmodnet : Subnet 192.168.118.0 is not a configured subnet.
cmmodnet : Use the "netstat -in" command to list the configured subnets.
ERROR: Function remove_ip_address
ERROR: Failed to remove 192.168.118.51


I feel this is a misleading message because if there was to be an issue with the subnet, the primary node would not bring the package up in the first place.

Can anyone point me in the right direction.

Alex R

[Jim Carter]

It looks like you may have assigned the same address to your static IP and your virtual IP. These must be different and you must have a different static IP for each node. The virtual can float between NICs or nodes, but can't be the same as a static.

It doesn't hurt to run Heartbeat on all (up to 5) LANs, but don't configure an interface for Heartbeat only and then try to put a virtual IP on top of it.

If this doesn't point you in the right direction, please let us see what your IPs are and how they are assigned.

[Stephen Doud]

Hello Alexis,

The package log contains a message you need to heed:

cmmodnet : Subnet 192.168.118.0 is not a configured subnet.

Perform a "netstat -in" and see which lan NIC supports that subnet address. If you don't see the subnet 192.168.118.0 assigned to a LAN, it cannot be referenced in the package control script.

-s.

[Sanjay_6]

Hi Alexis,

Do you have a network card configured on the other node in the 192.168.118.0 subnet.

Hope this helps.

Regds

[Alexis_2]

All,

Sorry for the lack of info. Here is an output of netstat -in:
NODE 1=
Name Mtu Network Address Ipkts Opkts
lan1 1500 192.168.113.0 192.168.113.10 65088 34178
lan0 1500 192.168.118.0 192.168.118.10 8105381 85165
lo0 4136 127.0.0.0 127.0.0.1 41087 41087

Node 2=
Name Mtu Network Address Ipkts Opkts
lan2 1500 192.168.118.0 192.168.118.11 251573 2672
lan1 1500 192.168.113.0 192.168.113.11 289 326
lo0 4136 127.0.0.0 127.0.0.1 6312 6312

Here is the way my cmclconf.ascii file is configures for the cluster:

# Definition of nodes in the cluster.
# Repeat node definitions as necessary for additional nodes.

NODE_NAME neldbs2
NETWORK_INTERFACE lan0
HEARTBEAT_IP 192.168.118.10
NETWORK_INTERFACE lan1
HEARTBEAT_IP 192.168.113.10
NETWORK_INTERFACE lan2
FIRST_CLUSTER_LOCK_PV /dev/dsk/c4t0d1
# List of serial device file names
# For example:
# SERIAL_DEVICE_FILE /dev/tty0p0

# Primary Network Interfaces on Bridged Net 1: lan0.
# Possible standby Network Interfaces on Bridged Net 1: lan2.
# Primary Network Interfaces on Bridged Net 2: lan1.
# Warning: There are no standby network interfaces on bridged net 2.

NODE_NAME neldbs3
NETWORK_INTERFACE lan2
HEARTBEAT_IP 192.168.118.11
NETWORK_INTERFACE lan0
NETWORK_INTERFACE lan1
HEARTBEAT_IP 192.168.113.11
FIRST_CLUSTER_LOCK_PV /dev/dsk/c4t0d1
# List of serial device file names
# For example:
# SERIAL_DEVICE_FILE /dev/tty0p0

Could I be getting this error because the heartbeat IP is the same as the interface IP address? If so, how can I address this?

Thanks again.

Alex

[Alexis_2]

All,

I think I found out what the issue was with a bit direction from Jim.

I reconfigured the cluster to contain one heartbeat ip lan rather than two. The other lan I configured as stationary IP. When applied all my changes and brought up the cluster, the package failed over correctly as designed.

This means that the HP documentation that I have is possibly incorrect because my manual states (H6487 Ver H.00 Section 5-4)
"Lan cards configured as HEARTBEAT_IP are used for heartbeat traffic and potentially for data traffic"

Under my configuration, both lans were configured as Heartbeat lans with the idea that data traffic (other than heartbeats) can go between them and it did not work. The system was unable to modify the ipaddress since it was considered a heartbeat lan.

Thank you all for your input.

[Duncan Edmonstone]

Alexis,

I'm not convinced this was the source of your problem - I 've implemented a number of MCSG clusters and I *always* ensure that all LAN cards are used for heartbeat (you can't beat that extra redundancy!), but have nvere had the kind of failover problems you describe here. What version of MCSG are you running? Do you have the latest patches installed?

HTH

Duncan

[Alexis_2]

Duncan,

I'm glad to hear that. I too was also a bit uneasy but I did not want to leave the folks here up in the air. And, since I had not picked anything else up from the forum (related to this item or anything close), I thought I should post my findings in lieu of further input.

SYSTEM

I'm running HPUX 11.0 in 32 bit mode on two H-60 Servers. Here is my version of Service Guard as well as the patch level:

B3935DA A.11.13 MC / Service Guard
B8325BA A.01.02 ServiceGuard Manager
XSWHWCR1100 B.11.00.54.6 HP-UX Hardware Enablement and Critical Patches, September 2001

I'm aware that there are several updated patch versions but I'm unaware of any updated patches for Service Guard.

Thanks for looking into this with me.

Alex

[Jim Carter]

Alexis, you should be able to run heartbeat on every lan (up to 5) since it is a broadcast activity and first come / first serve answers the "how do I feel question" just fine.

Is there a possibility that you didn't have the 192.168.118.0 lan configured when you tried to bring up the package initially? That would also cause the problem.

It might help to see the syslog output just prior to, during, and immediately after your package failed to start. Don't look for MC/SG issues, since this is probably LAN based instead.

[Steven Hargus]

Alexis -

"Lan cards configured as HEARTBEAT_IP are used for heartbeat traffic and potentially for data traffic"

You're right, it's a little misleading, although not entirely wrong. Yes, you can use your heartbeat LANs for data, as long as ServiceGuard is not managing that data. It will not use a heartbeat NIC as a primary or failover data network, but you can do (non-critical!) data transfers over the heartbeat, if you want. I think what they had in mind was that, since there is normally very little traffic over the heartbeat, you can make use of that network for other, occasional uses (like copying small files.) Be aware that, if you saturate your heartbeat network, you risk having your cluster fail over erroneously. Your best bet is to dedicate your heartbeat LANs for just that purpose: heartbeats. Ideally, you would want two heartbeat networks (for redundancy) and two data networks (again, for redundancy). Furthermore, you should never have your primary and redundant nets on the same card, since then the card becomes a SPOF.