TruCluster won't boot (quorum disk in use)

[DELAHAYE_2]

Hello.
I create a T64 cluster 5.1b-4 with DEGXA for the cluster interconnect.
When the 2nd node try to boot It can't communicate with the first node with DEGXA and wait for the Quorum disk to be available.
After some internet search I find someone who had the same problem.

(I don't have this problem if the first node boot after then 2nd one)

http://unix.derkeiler.com/Mailing-Lists/Tru64-UNIX-Managers/2004-06/0004.html

I want to use the same solution but how to change a cluster Lan card (DEGXA to DE600) without destroy the cluster

[Hein van den Heuvel]

Bonjour Mr Delahaye,
bienvenue aux Forum Tru64.

To begin with the end:

>> want to use the same solution but how to change a cluster Lan card (DEGXA to DE600) without destroy the cluster

There is not such thing as a cluster lan card. The clusterinterconnect is defined by IP name/number, not by interface. That name/number is ofcourse associated with an interface, but can be freely moved.
Check with clu_get_info.

>> When the 2nd node try to boot It can't communicate with the first node with DEGXA and wait for the Quorum disk to be available.

That would be correct. If the quorum disk is already attached by the first member to create a cluster, then an other should not and can not use that quorum disk without communitation with the originating mode. Otherwise a split cluster would occur. Not good.

What are the details on that 'can't communicate'.
How do you know? What error message(s)? Where?
Can you boot standalone and check out the network without trying to be a cluster member? (ping...)

>> (I don't have this problem if the first node boot after then 2nd one)

That's somewhat hard to believe. Communication problems then to be symetrical. If a and not talk to b, then b can not talk to a. Still, this may also be a big helper to find what might be wrong.

I would treat this as a network problem, not a cluster problem. How are the systems connected? Switch? DNS? DHCP? Static addresses for the cluster interconnect I hope? Area 10.x.x.x?

Good luck!
Hein van den Heuvel

[Rob Leadbeater]

Hi,

Can you confirm exactly what happens when the system tries to come up...

I've seen a second node not join the cluster after a reboot if the SRM variable boot_reset is set to OFF.

Cheers,

Rob

[Vladimir Fabecic]

Hello DELAHAYE
Can you supply some more informations about configuration?
You want to change cluster LAN card?
You want cluster card to be DE600 (fast ethernet) and not DEGXA (gigabit ethernet)?
I would not recommend that. When both nodes use same file systems actively cluster inetrconnect traffic may be heavy.
OK, everything depends on how cluster is projected.
What will this cluster be used for?
DEGXA is Broadcom chip based card.
Is it copper or fiber optic card (3X-DEGXA-TR or 3X-DEGXA-SR)?
And yes, it is possible to change cluster interconnect LAN card (without destroying the cluster). But you must know what you are doing.
First, please, post your /etc/inittab file.

[Ivan Ferreira]

I have seen this problem several times before. It's a problem with the autonegotiation of the cluster interconnect network adapter.

How do you connect your cluster interconnect? To a switch? Crossover cable (not supported)?

You can do this.

Use the lan_config command to set the speed of the network adapter in the /etc/inet.local on both servers:

/sbin/lan_config -i bcm0 -s 1000 -x 1 -a 0

This will force 1000 FD.

When the second node fails to boot, run that command on the first node. You can also reduce the speed to 100 FD for troubleshooting.

The real problem is that you cannot disable autonegotiation for bcm interfaces, but the autonegotiation does not work correctly sometimes.

[Rob Leadbeater]

Note that if you do this:

> /sbin/lan_config -i bcm0 -s 1000 -x 1 -a 0
>
> This will force 1000 FD.

as suggested by Ivan, then you must, stress must, also set the switch port to be fixed at 1000 Full Duplex.

If one end of the link is fixed, and the other set to auto-negotiate, you *will* get a duplex mismatch.

Cheers,

Rob