Storage Boards Cleanup
To make it easier to find information about HPE Storage products and solutions, we are doing spring cleaning. This includes consolidation of some older boards, and a simpler structure that more accurately reflects how people use HPE Storage.
HPE StoreVirtual Storage / LeftHand
cancel
Showing results for 
Search instead for 
Did you mean: 

HP 4500 G2 14.4 Configuration Theory

L1nklight
Valued Contributor

HP 4500 G2 14.4 Configuration Theory

Hey everyone,

I am currently working with the HP 4500 G2 (14.4 Tb version). I have a pair of them so I hope to be setting up redundancy. Just for clarification, I have a total of 4 HP 4500 G2 shelves, each one containing (12) 600 gig SAS drives. I am mostly familiar with the DS series from IBM as I have worked with those for about 6 years and I am rather new to the 4500s. These storage devices will be providing storage to our VMware environment (vSphere 5). While a  majority of our I/O operations will be relegated to WebApp work we will have some SQL servers planned out. Whether or not they will use RDMs directly from the 4500s or if they will simply use vmdk files cut from existing VMFS volumes is yet to be seen.

 

I have questions about best practices for configuring the 4500 specifically. For example, those of you with similar configurations: Do you use a mix of RAID 5 and RAID 10? Is it necessary to consider RAID 10 when you use and configure network RAID (for redundancy)? Holding spindle count the same, do you see signficant I/O improvements between RAID 5 and RAID 10 on these devices (performance)?

 

Thanks in advance and if there is some thought provoking feedback I may have other questions.

 

Edit: Additional Question! Can a single HP 4500 only be configured with one type of raid setting across all 12 disks? For example, I could I build a 6 disk raid 5 array and an 6 disk raid 10 array on the same box? If I needed to mix RAID configurations would it be best to form a single cluster with the mixed environment or should I form 2 clusters, one raid 5 array and one raid 10 array cluster?

6 REPLIES
Paul Hutchings
Super Advisor

Re: HP 4500 G2 14.4 Configuration Theory

AFAIK you can only have one physical RAID type per node, even though the P4500 has two RAID sets - reason being you can't choose which individual node/RAID to put a volume on.

 

As for the VMDK vs. RDM, have you ruled out mounting the SQL DB/Log volumes from within the SQL guest?  You'll gain from being able to use the MPIO DSM plus if you decide to use them you'll also be able to use application manages snapshots.

L1nklight
Valued Contributor

Re: HP 4500 G2 14.4 Configuration Theory

No actually, I haven't ruled that out at all. It will be completely dependant upon the application's use of SQL. The application is new and untested. If I need a crushing amount of IOPS then maybe RDM > virtual machine would be necessary but if the application uses IO on the scale of like 3-5k IOPS for the SQL side of things it looks like (according to this data: http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA1-4478ENW.pdf) that can be easily handled by both a RAID 5 and RAID 10 configuartion. What I was thinking of doing was to setup one node/cluster with a RAID 5 configuration for the bulk of my virtual machines, which would be very cost effective and setup the other node/cluster with a raid 10 configuration for those apps that may need it. The raid 5 node would have about 6 TB of usable storage and the raid 10 configuration would have about 3.2 TB of usable space. I was thinking something like this potentially:

 

Management Group A

Node A (Raid 5)<(Network Raid)>Node C (Raid 5)

 

Management Group B

Node B (Raid 10)<(Network Raid)>Node D (Raid 10)

 

Each node consists of 12 600 gig 10k SAS drives. Management Group A would have 6 Tbs of usable space and Management Group B would have 3.2 Tbs of usable space. Does this configuration make sense? Am I using the right vernacular?

 

By the way, I really appreciate the response. Thank you for bringing some new ideas to the table for me.

Emilo
Trusted Contributor

Re: HP 4500 G2 14.4 Configuration Theory

Stay away from RAID 5 its not worth he disk space saved for the performance trade off. Saniq was designed to work with network RAID 10 and performs very well and provides redundancy. You can change the hardware level to a different level then RAID 5 but its not considered best practice. I think the previous post makes good point about using MPIO with the SQL box but you cant make it a virtual machine. There are alot of benifits to using MPIO and it is fairly easy and straight forward to setup, remember the more spindles you have the better the performance Saniq really starts to shine after nodes.Here is a document on best practices of using VMware and the P4000 series.

http://www.vmware.com/files/pdf/techpaper/vmw-vsphere-p4000-lefthand-san-solutions.pdf

Good Luck.

 

 

bryan_hadzik
Frequent Advisor

Re: HP 4500 G2 14.4 Configuration Theory

Lets clarify a few terms first. P4000 has some unique things about it that aren't the same on other sans.

 

There are 2 different "RAID" levels you need to deal with.

 

P4500G2 15kSAS

1. Hardware raid on each individual node. However, every unit in the cluster must be the same.

a. RAID 5.

             - two raid 5 sets of 6 disks each

     - Default setting. 95% of customers use this setting

     - 1700 IOPS per unit

b. RAID 6.

             - two raid 6 sets of 6 disks each

             - 1250 IOPS per unit

     - I recommend against this except for extreme situations. too many drawbacks

c. RAID 10.

             - One raid 0 set on 6 disks, then striped across the 2nd set of 6 disks

             - 2400 IOPS per unit

2. "Network RAID" (Previously called "replication level")

 This is set on a per volume basis, and dictates how many times we write the block

 

a. Network Raid 0

     - Block is only written one time

      - 1:1 space consumption

     - No extra protection over hardware raid. if one unit fails, all the data is lost/unavailable

     - normal speeds on reads/writes

b. Network Raid 10

       - Block written twice, cascading down the stack

            - 2:1 space consumption(twice as much)

            - You can loose up to 50% of non-consecutive nodes

    - slightly slower writes/faster reads

c. Network Raid 10+1

        - Block written three times, cascading down the stack

            - 3:1 space consumption(three times as much)

            - You can loose up to 66% of non-consecutive nodes

   - slightly slower writes/faster reads

   - very rarely used

 

c. Network Raid 10+2

        - Block written four times, cascading down the stack

            - 4:1 space consumption(four times as much)

            - You can loose up to 75% of non-consecutive nodes

       - slightly slower writes/faster reads

    - Needs at least 4 nodes in cluster

    - very rarely used

 

d. Network Raid 5

        - Block written 3+ parity, cascading down the stack.

            - 1.25:1 space consumption(25% overhead)

            - You can loose up to 25% of non-consecutive nodes

            - Only good with mostly read/sequential data

            - Needs at least 4 nodes in cluster

 

e. Network Raid 6

        - Block written 3+ parity, cascading down the stack. Double parity

            - 1.50:1 space consumption

            - You can loose up to 50% of non-consecutive nodes

            - Only good with mostly read/sequential data

            - Needs at least 4 nodes in cluster

 

90% of customers use hardware raid 5, and network raid 10 for their volumes.

 

Now with that out of the way, lets talk about the dual cluster model you mentioned. I prefer all in one big stack, and here is why:

 

If you have one raid5 cluster, and one raid 10, the iops will be 3400, and 4800 respectively. If you keep it all in one cluster, you will have 6400. I prefer having a larger bucket of iops together, that apps can burst against. In the 2 cluster model, you might max out one cluster, but the other one could be low, and going to waste.

 

L1nklight
Valued Contributor

Re: HP 4500 G2 14.4 Configuration Theory

Bryan, thanks a bunch for the info. Your post by far has been one of the more helpful things I've read about the P4000 system (and subsequently my 4 P4500 G2s). One thing I am uncertain about is how the additive property of IOPS works with a combination of the Hardware RAID and Network RAID levels. If each unit gives you approximately 1700 IOPS (which makes sense with 15k SAS drives; 150 IOPS ea. * (12 -1) = 1650) then the network RAID configuration is somehow additive. It's completely clear how this setup is resiliant but the additional IOPS is confounding me.

 

Bryan, how often have you seen a requirement of over 6400 IOPS? My worry is that we are building this system for a new application for our environment and we haven't been given any sort of IO requirements. Seemingly it is just a web app with a SQL back end and all of this will run inside of VMware, but I used to deal with some crappy code and I've seen re-index jobs on SQL servers hit the 10-11k high water mark and that was with an IBM SVC. 

 

FInally, it seems as if you are pushing me towards a hardware RAID 5 and a network RAID 10 setup. With each node holding holding about 6 tb and a total of 4 nodes, what does that make my end of the day usable storage space? If I am doing my calculation correctly looks to be about 12 Tbs or so?

 

Hey Emilo, thanks for your input too! 

bryan_hadzik
Frequent Advisor

Re: HP 4500 G2 14.4 Configuration Theory

We stripe the data accross all the units in a cluster, one block at a time. So while we ask node 1 for block A, before it responds, we can ask node 2 for block B, and so on. That was we can have all of them working at the same time for read requests. The entire slowdown in the process is the actual disk physically moving. All the other peices are very fast. Memory/network/cpu. The disk is 10x slower compared to these peices. Spreading the load out is the most efficent way.

 

6400 IOPS is a lot for one app. SQL can potentially use it, but just make sure to follow the best pratice of spliting up log files and db files, things like that. All these IOPS are theoreticle. If you want to see something closer to reality(still just a benchmark though) use iometer: http://iometer.org/

 

Here are the real numbers on capacity.

Each node after RAID 5 has 5,310 GBs

That means a total cluster capacity of 21240

Lets assume you do network raid 10 on every volume (Remember, you can change it per volume. make test data network raid 0) that cuts the number exactly in half, since we write the blocks twice:

 

= 10,620 GBs