topic Node failure in P4000 SAN cluster in StoreVirtual Storage

Node failure in P4000 SAN cluster

jswest — Sun, 15 Nov 2009 19:32:13 GMT

I know that if one entire node fails in a cluster and the volumes have been configured with R2 then the data will still be available. Is there a level of redundancy that would allow two nodes to fail like local raid 6 does?

Re: Node failure in P4000 SAN cluster

thomas kiblin — Mon, 16 Nov 2009 01:00:45 GMT

Jamie,

You can set the # of copies to more then 2, but it cannot be more than the # of nodes in the cluster.

Example, 4 nodes in the cluster, you can have 4 copies, and it will keep 1 copy on each node.

Tom

Re: Node failure in P4000 SAN cluster

jswest — Mon, 16 Nov 2009 01:12:03 GMT

Thanks Tom

What I was looking for is how many nodes can fail. And it turns out that even with Raid 2 you can lose 2 non concurrent nodes in a 4 node cluster and with Raid 3 and 4 you can lose concurrent nodes.

Jamie

Re: Node failure in P4000 SAN cluster

Niels Vejrup Pedersen — Mon, 16 Nov 2009 05:40:53 GMT

Hello,

By using Network Raid 2 in a two node cluster it will depend on which nodes fails for the volume to be available.

Another thing to take into consideration is the Quorum - it's based on majority node set - so if you loose 2 nodes in a 4 node cluster, you won't have quorum, and the whole cluster is down.

This can be solved by implementing failover managers.

Regards

Re: Node failure in P4000 SAN cluster

jswest — Mon, 16 Nov 2009 14:21:02 GMT

Niels

If network raid is set to 3 or 4, not 2, shouldn't you be able to lose 2 nodes and still be able to access the raided volumes?

Jamie

Re: Node failure in P4000 SAN cluster

Mike Povall — Tue, 17 Nov 2009 17:14:24 GMT

Hi Jamie,

If you have a 4 node cluster and your volumes are 3 way replicated then you should be able to lose two nodes assuming that you have a failover manager running. This will enable you to run five managers and when the second node fails you will still have three managers running thus maintaining quorum and the volumes will still be accessible.

Mike.

Re: Node failure in P4000 SAN cluster

Jason.Wilkening — Wed, 18 Nov 2009 03:44:18 GMT

I think what others are trying to say, is it depends on your cluster configuration. Obviously if you want the ability to lose two or more nodes then you will need more than 2 nodes. Like others have said if you have 3 way replication then yes you can lose more than 1 node and be safe.

Also remember that if there is a 4 node multi-site cluster, then you can lose a whole site, or two nodes. And you only have to have 2-Way replication for that.

Then there is the case where you lose non-consecutive nodes in a multinode cluster and should be safe. Again with only 2-Way replication. Take a look again at how the blocks are written across the nodes and that should show you how you can lose multiple nodes as long as they are not consecutive.

Hope this helps show that the real answer is it depends. Also like others have stated make sure that you maintain quorum as well. Losing quorum doesn't cause data loss, it just causes the cluster to shut down and access to volumes is stopped.