P4500 local and network RAID clarification sought

[sls]

I have recently become the owner of some P4500G2 storage systems and I am trying to understand how data is written across the drives and exactly what failures can be tolerated without disrupting the service. Each of the storage systems has 12x 600GB SAS drives and the default RAID setup (from the Storage/RAID Setup view in CMC) indicates that each controller has its own RAID 5 array with 6 drives. Does this mean that data is striped across 6 or 12 drives? Can I afford to lose one drive on each controller without data loss (ignoring Network RAID momentarily)?

I will be using 4 node clusters and I would also like to understand the relationship between the local RAID 5 within the storage node and the Network RAID level that might be used. In a Network RAID 10 environment, is the data striped across a single storage system and then effectively mirrored to another single storage system? If so, how is the partner storage system selected out of the other three possible candidates? 

Any thoughts would be appreciated!

[Dirk Trilsbeek · Accepted solution]

The single nodes are by default configured with 2 Raid 5-Volumes, each using 6 drives. So yes, you can lose one disk on each volume without losing the node.

There is no connection between the local raid and the network raid. Local raid only protects the single node, so that you don't lose that node when a disk is faulty. Network raid keeps your volumes available in case a node fails.

Network Raid 10 means that all blocks are written to 2 consecutive nodes. "Consecutive" means exactly that - the order the nodes appear in the CMC is the order the data is mirrored over 2 nodes. All volumes are spread over all nodes of a cluster (that's the Raid 0 part), the network raid then defines if and how a volume is protected from failure of one or several nodes. Network Raid 10 is a 2-way-mirror (each block is written twice), you can also create Network Raid 10+1 (each block is written 3 times) or Network Raid 10+2 (4 times). There also is Network Raid 5 and 6, but i never used them, so i won't try explaining them.

Due to the way the nodes mirror blocks, you can lose up to 50 % of your nodes without losing access to your Network Raid 10 volumes, under the premise that the nodes you lose aren't direct neighbours. A multisite setup for instance makes sure that nodes from site 1 are placed alternating with nodes from site 2, so that your cluster is protected from failure of a complete site.

In any case you should run a Failover Manager to provide quorum. To avoid a split brain situation, the P4000 makes all volumes unavailable that don't have quorum (majority of votes, or "managers", in lefthand terms). In a 4-node-setup, losing 2 nodes would give you exactly 50 % of all available votes - not a majority. A distinct 5th node (the failover manager) that usually runs in a hyper-v or esx environment provides that majority.

[Steve Burkett]

Frank Denneman had a pretty good write up (with pictures!) on Network RAID operations over on his blog here.

[AUSSUPPORT]

HI,

I got 2 x 4500 and 1 x VSA for DR site.  Third Party set this for us. and got few questions 

•  Raw space is 6.55TB per node.
• Useable space is 5.34TB per node (based on RAID5 calculation of the raw space).
• Since both nodes are configured for redundancy (Network RAID 10), allocate-able space is 5.34TB across both nodes.
• From this, 2x 2TB LUNs were provisioned, leaving 1.34TB free.

1. Is this the better way of setting up this SAN? 

2. Look like we lost 10TB space doing  network raid?

3.  Do we need snapshot volumes?

4. do we need FOM?

5. crash of 1x P4500 what happen?

6.  How this remote copy works with VSA?

AS

[sls]

Steve/Dirk - many thanks for your responses and apologies for my tardiness in replying!

[Dirk Trilsbeek]

@AS

1: i would consider this the best practice. Both nodes and cluster are protected from failure of a single member

2: you lost some space due to Raid 5 on the local node and then you halved your remaining space by using a mirror over 2 nodes. So you essentially lost around 7,76 GB, yes. With redundancy that can't be totally avoided.

3: depends on what you want to do. They can be helpful for backups, but we never used them.

4. Yes, you need an uneven number of managers in your cluster to have a quorum, especially with only 2 nodes in your cluster

5: without FOM, volumes in your cluster are unavailable. With an FOM present and running, the volumes stay up.

6: never used it, so i can't comment on that feature.