Second controller

[Deborah Weatherspoon]

Hi all,

We have two L3000 which should failover if there is a problem. However, yesterday we had a bad GBIC on one controller but both systems were down ??? They only replaced one GBIC and then everything was fine after changing the world wide name??? We have two controllers

Can anyone help me understand why didn't the systems failover? Is there anything I can do to prevent this next time ?

Thanks

[TTr]

I assume by failover you are using serviceguard and that the GBIC was on a storage controller and not LAN.

A bad GBIC does not warrant a service failover. In a critical environment as with your cluster, you would have set up dual path and load balancing for all your storage.

A single GBIC would NOT bring any of the servers down. Even the server with the bad GBIC should stay up and there should be NO failover. Something else is/went wrong or your server/storage setup is not right.

[Torsten.]

You should give more information about your hardware. Where is this GBIC? On the switch or on the storage array? What kind of array?
I guess you will find more information in your syslog and cluster log files.

[Deborah Weatherspoon]

We have a va7100. There is a fiber cable that is in the brocade switch and the controller on both systems. I must add that we have serviceguard and while vg00 mounted I had to manually activate the volume groups for serviceguard. This is something we never had to do.

[Torsten.]

I assume the bad GBIC was on the VA controller. Does the array have dual controllers?

[Deborah Weatherspoon]

Yes

[TTr]

> We have a va7100. There is a fiber cable
> that is in the brocade switch and
> the controller on both systems.

Is there one cable that from the switch to each of the L3000?

What about from the array to the switch? one or two cables?

What about the zones in the switch, how are they setup?

The VA7100 is active/passive with only one redundancy group (failover only not load balancing)

> I must add that we have serviceguard
> and while vg00 mounted I had to manually
> activate the volume groups for
> serviceguard. This is something we
> never had to do.

Are you booting each server from the SAN?

[Torsten.]

Please don't call the VA active/passive!

From the manual:

"There are two internal fibre channel loops, one from each controller. The Fibre
channel disks are dual ported; each fibre channel port is connected to a
different controller. The controllers are connected via an internal highperformance
bus, which allows the LUNs to be accessed through both
controllers, and for loop or disk failover communication."


(Users manual - page 40)

[Torsten.]

I'm still not sure about your configuration.
Can you post the results of:

# armdsp -a
# armtopology

[TTr]

I agrre with Torsten, I don't consider the VA7100 active/passive either :) but HP does. When I confronted them at one point they claimed one controller is active the other is passive!!!

I guess vendors manipulate the terminology to their advanage.

From that point on I started to clarify with each vendor what they mean by active/passive by verifying that "the array is active/passive on both controllers" or "each controller is active with the other one being passive".

[Torsten.]

You are right, terminology is a world of it's own and can create real trouble.

Anyway, the VA is a HP classic and just like all other (original) hp arrays active/active.

This means both controllers are able to access disks/LUNs at the same time.

[Deborah Weatherspoon]

Here is output:

[Torsten.]

Your array could take some maintenance.
The firmware is old (HP19 vs. HP22), the disk firmware is likely old, the battery charger fw is different (!), I guess commandview is old too.

But this is not the current problem (but the command output you posted is not like expected).

Can you post "ioscan -fn" and "vgdisplay -v" to complete the picture?

[Deborah Weatherspoon]

Here's the output


Thanks for your help.

[Torsten. · Accepted solution]

What GBIC was bad?

Your vg03 does not have an alternate path!

You need to
vgextend /dev/vg03 /dev/dsk/c8t1d5
and sort the pathes.
(vgreduce the c10t1d5 and vgextend again to make it the alternate) - make sure to transfer this information to the other node too.

I assume switch port 18 is connected to controller 1 of the VA - this not clear to me yet.



This server can access the vg03 only via 1 path. If this fails, there is no access.

Check the other nodes "vgdisplay -v" output for "alternate links"!

[Deborah Weatherspoon]

Thanks Torsten.... I'll vgextend

[Deborah Weatherspoon]

I will have to vgextend... The members were great in answering my question....

[Torsten.]

You are welcome!

Just an additional note.
Check the output of
# strings /etc/lvmtab
You should see c8txdy and c10txdy for each vg (except vg00).
If not, adjust it.
I assume you are aware about the procedure how to transfer the LVM config from one cluster node to the others.