HPE 3PAR StoreServ Storage
1748104 Members
4723 Online
108758 Solutions
New Discussion юеВ

Re: Clarification on CPG HA Availability and amount or drive enclosures

 
Koobster
Occasional Contributor

Clarification on CPG HA Availability and amount or drive enclosures

Trying to get my head wrapped the CPG HA availability and the 3PAR chunlets concept.

  • HA CAGE means that no two members of the same RAID set can be in the same drive enclosure.

Lets say we have 4 enclosures and CPG of RAID 5 3+1 and choose cage availabiltiy. How is the chunklets get striped exactly if "no two members of the RAID set can be in the same drive enclosure"? One chunklet on one drive then the next chunklet on the next drive on a different enclosure , then next drive next enclosure and so on?

  • HA MAG means that no two members of the same RAID set are in the same drive magazine.

Lets say we have the same conecpt as above, 4 enclosures and CPG of RAID 5 3+1. Does a chunklet get striped across one chunlet to one drive at a time through the enclosure then vertical to the next drive enclosrue? 

 

ALSO: If we have a 4 node system and 2 additional drive enclosure, does that mean the total of enclosures is 4? (each node pair with a row of drives + the 2 additional enclosure)

5 REPLIES 5
yapkl
Trusted Contributor

Re: Clarification on CPG HA Availability and amount or drive enclosures

Hi koobster,

It's not exactly chunklets, further down each chunklets you have LDs. Multiple LDs come together to form the R5 3+1 that you have configured. So each of these 3D+1P LDs will be spread across disks from different enclosures.

As for HA MAG, that only applies to older systems where we have multiple drives in one magazines. On new 3PAR 7000/8000/2000 one magazine = one drive so HA MAG can be understood as HA "DRIVE". No two LDs from the same RAID set size can be on the same drive (if that drive fails, that raid set will fail too).

Each node pair will have their own drive cages, so if you want to do R5 3+1 HA CAGE, you will need 4 enclosures per node pair. Meaning node0/1 will have 3 additional drive enclosures (the node itself is an enclosure) and node 2/3 another 3 additional drive enclosures. Total enclosures (including nodes) will be 8.

This document will have more answers and details to your questions, see page 12.

https://www.hpe.com/h20195/v2/getpdf.aspx/4aa3-3516enw.pdf

 

 

Best Regards,

yapkl

Note: The opinions & comments above are my personal opinions, not of Hewlett Packard Enterprise
----------
If my post was useful, click on my KUDOS! thumb below! Thanks.
Koobster
Occasional Contributor

Re: Clarification on CPG HA Availability and amount or drive enclosures

It's not exactly chunklets, further down each chunklets you have LDs. Multiple LDs come together to form the R5 3+1 that you have configured. So each of these 3D+1P LDs will be spread across disks from different enclosures.

I think im starting to understand more. So if its R5 3d+1p, the chunklets in each enclosure will wide stripe to form a set of LDs. So 3 enclosures will have its own Data LD and the last enclosure will be the Parity LD.... assuming its R5 3D+1P.

 

 Each node pair will have their own drive cages, so if you want to do R5 3+1 HA CAGE, you will need 4 enclosures per node pair. Meaning node0/1 will have 3 additional drive enclosures (the node itself is an enclosure) and node 2/3 another 3 additional drive enclosures. Total enclosures (including nodes) will be 8.

Without taking RAID / Availability to count each node pair is a drive enclosure? Like node 0/1 has its own enclosure with 24 SFF disk slots in the front. and node 2/3 the same. So without any more additional disk sheleves I have a total of 2 Disk enclosures? And if i were to buy an additional 2 disk sheleves to the 4 node system the total would be 4.

Sheldon Smith
HPE Pro

Re: Clarification on CPG HA Availability and amount or drive enclosures

A chunklet is a contiguous area of a single physical disk. On earlier systems (F-class, T-class and before) chunklets are 256MiB. On all StoreServ systems (7000s, 10000s and later) all chunklets are 1GiB.
Chunklets are micro-JBODs - they have no RAID.
CPGs direct the system how to build RAIDsets. Those RAIDsets are wide striped to become Logical Disks.

You mention you have a four-node system with two extra cages. Cages are connected to a single node pair. You have two node pairs, presumably balanced, so each node pair has one additional cage. The showcage command will list each cage and to which node pair each cage belongs, 0&1 or 2&3.
RAIDsets are created within a node pair and its cages. You can have "RAID1 set size 2 -ha cage". Anything larger involving 3 or more chunkets (RAID5 2+1, RAID1 set size 3) will be limited to "-ha mag" since each node pair has only one additional cage.
If you want RAID5 3+1 (set size 4) and HA Cage, you would need four (4) additional cages - two additional cages for each node pair. Of course you would also need sufficient physical disks to "stretch" the columns of disks from the two existing cages (node pair and add-on) to the four new cages.


Note: While I am an HPE Employee, all of my comments (whether noted or not), are my own and are not any official representation of the company

Accept or Kudo

yapkl
Trusted Contributor

Re: Clarification on CPG HA Availability and amount or drive enclosures

and here's a very good explanation on how 3PAR virtualizes the disks into chunklets and then LDs to create volumes for your hosts.

https://d8tadude.com/2014/10/28/3par-chunklet-raid-architecture/

Best Regards,

yapkl

Note: The opinions & comments above are my personal opinions, not of Hewlett Packard Enterprise
----------
If my post was useful, click on my KUDOS! thumb below! Thanks.
Dennis Handly
Acclaimed Contributor

Re: Clarification on CPG -ha Availability and amount of drive enclosures

> Does a chunklet get striped across one chunlet to one drive at a time through the enclosure then vertical to the next drive enclosure? 


It could but doesn't have to do that.

 

> does that mean the total of enclosures is 4?

 

It depends.  For 7000/8000, the node pair also includes drives.  For 10000/20000 they don't.

So no need to ask as showcage will show you how many cages.  :-)

 

> further down each chunklets you have LDs

 

No, further UP.  :-)  VVs include LDs.  LDs include various PDs for raidsets.  At the LD and PD level, they deal with chunklets.

 

> Multiple LDs come together to form the R5 3+1 that you have configured

 

No, multiple chunklets across PDs.  LDs are how the VVs are spread across the nodes.

The Concepts Guide will also help:

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=c04204225