Serviceguard internal lan failover - device busy

[Shawn M Harris_1]

My problem occurs at this point during testing of internal lan failovers. We are running on 32-way Superdomes with redundant 1000-SX cards (GigE). When an interuption in connectivity to the primary card fails Serviceguard correctly begins the monving of the hard IPs and the VIPs to the other card. lan1 is the primary and lan6 is the failover lan interface. Frequently the we receive a failure in failing the IP to lan6 with a device busy message. I have bandaided this in the past by stopping the cluster services on the node and restarting the cluster daemon. Obvisouly this is not a fix but a temporary work a round. Recently this did not fix the problem either. Currently 90% of the time the internal failover does not occur.

[Sridhar Bhaskarla]

HI Shawn,

Do you have the failover card (lan6) configured by any chance in your /etc/rc.config.d. If there is any references to it remove. I believe the failover card's administrative status should be in 'down' state. Try the following.

#ifconfig lan6 down

pull out the cablel connected to lan1 and see if it will failover.

Make sure there is no IP Address configured to lan6.

-Sri

[Shawn M Harris_1]

I did confirm prior and post pulling the cable that lan6 was not configured within /etc/rc.config.d/netconf. The only locations configured for lan6 was in the cluster ascii file. I do have to VLAN interfaces on the card as well. So I do have lan interfaces defined to reside on lan6.

In /etc/rc.config.d/vlanconf I have lan5075 configured as a VLAN on lan1 and lan7075 as a VLAN on lan6. This does fail the VLAN interfaces from one card to another.

My main difficulty in wrapping my head around this on is that each test has a different result. Sometimes lan1 IP will failover to lan6 and the 2 VLANs will not. Sometimes the opposite, and sometimes a VLAN and lan1 will failover.

What could I use to find out what the system believes has the device in use. Is lsof an option? If so I have limited experience with lsof. Any suggestions?

[Sridhar Bhaskarla]

Hi Shawn,

I haven't worked on VLANs yet. So, I am afraid I don't have an exact answer. But I believe this behaviour has something to do with how your VLANs are setup and configured in your serviceguard. I suggest you go through the following document

http://docs.hp.com/hpux/onlinedocs/B3936-90079/B3936-90079.html

and see if you have done anything differently than that is mentioned. Look at 'how the network manager works' section.

-Sri

[Shawn M Harris_1]

This problem resulted in a template from OpenView's OS-SPI (Smart Plugin). It was noted in the release notes and should have been seen at OS-SPI installation time, but the default behavior of one template within the SPI is to monitor all network interfaces and ensure they are ifconfig up state. It also has an action defined by default to ifconfig up the interface if found down. Well in Service Guard the standby interface cannot be configured or in an up state.

FIX: We remove the action to ifconfig up the interface, only to montor.