topic Re: Vdisk Size and Disk Spindles underneath on EVA 5000 in Disk Enclosures

Vdisk Size and Disk Spindles underneath on EVA 5000

rccmum — Thu, 11 Aug 2005 17:30:50 GMT

HI Guys,

EVA 5000 ( 2C6D ) 56 Disks
CV EVA 3.3
VCS 3025

I am curious to know about any way to find out the number of disk spindles or rather disks who
would underneath the Vdisk. Suppose I create a Vdisk with Vraid1 of size 50GB , how much disks would be involved under it?

EVA Best pratices says it's better to have few large vdisks than more small vdisks. I understand that this would improve performance of a Vdisk. One can judge the optimized size of a Vdisk if he knows how much spidles would be involved at the array side.

Points for everyone and full points for smart replies!!!!

Thanks in advance

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Susan Carleton — Thu, 11 Aug 2005 18:06:44 GMT

The question does not so much revolve around how many disks are underneath the vdisk, but rather how many disk groups have you set up with your 56 drives? The vdisks are striped as evenly as possible over ALL of the drives in the disk group you create it in. Therefore, if you have one disk group, the vdisk will be written across the top of ALL of the drives in the array.

You can reduce the number of disks in the vdisk by creating multiple disk groups, but you will get better performance and, ultimately, better redundancy, with more drives and less disk groups.

The main reasons for creating multiple disk groups would be the following:
1) You have FC drives and FATA drives. They cannot reside within the same disk group.
2) You have a database application that wants its logs on a separate set of spindles.

Hope this helps!

Susan

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Steven Clementi — Thu, 11 Aug 2005 21:43:48 GMT

Susan is correct in saying that you vdisk is striped across all of the disks in the Disk Group the Vdisk is created in.

To add some more to this...

HP recommends that for BEST performance, only 1 or 2 disk groups be used or "as few as possible". Putting more disks into 1 or 2 Disk Groups increases the performance of those groups because of the additional spindles.

The most cost effective configuration is to have 1 Disk Group with Double Sparring or even Single Sparring if you don't required the extra redundncy. Any additional Disk Groups would incur their own sparring cost reducing your total usable storage.

The EVA's "sweet spot" is at about 56 Disks in a single group. This means that you get the most performance gain when you add up to 56 Disks. After the 56th disk, you still gain performance, but not as much as you did when you added the 56th disk. This does not mean you should NOT make your disk groups only 56 disks, just don't expect the same performance increase from 80 disks as compared to 56 disks.

Having a few large vdisks vs. many small vdisks doesn't make much difference. It really depends on your application requirements. Keep in mind that only one server at a time can utilize a single vdisk (unless of course you have a cluster or some other software that provides disk locking so that as one server is accessing/writing, others can only read.)

Always create disk groups that contain an amount of disks that is a multiple of 8 and always add disks in multiples of 8. This allows RSS to not have to work too much when expanding and adding disks. The nominal number for RSS is 8 disks, it strives to keep sets of 8 when possible.

Steven

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

generic_1 — Fri, 12 Aug 2005 00:06:46 GMT

http://h18000.www1.hp.com/emea/presalessupport/downloads/EVA_Best_Practices.pdf#search='eva%20best%20practice'

This is the best reference.

Make small vdisks of the same size, say 10gig, and use lvmstriping.

If you are in shopping mode consider an XP or EMC DMX3000. More expensive, but they are more reliable and faster than the EVA5000. You have to take the whole frame down to do firmware, and it has no ups built in to the array, or active active controllers on the EVA5k.

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

rccmum — Fri, 12 Aug 2005 09:23:39 GMT

Susan , I Do agree with your statements.

If I am not wrong ,one can pin-point number of disks associated with a DG based on RSS ID and Index. It should be that way as there is physical separation of disks across the DGs. I am trying to find out this association using CV.I don't know how the VCS selects the disk spindles for a DG.

But DG is again using subsets as RSS which in turn contain 6 to 11 disk spindles!!! . I have two DGs having 40 and 16 disks in each group. This imples me to have 5 & 2 RSSs having 8 spindles each for the respective DGs. Now when we create a Vdisk with Vraid1 , two mirrored block or stripe will always reside on different disks but within the same RSS . I know this mechanisam is for redundancy against disk failure. My point is , there is a always an association of Vdisks with the spindles below them ( RSS ) . I didn't find a way to pin-point this using CV.

Steven,
I am curious to know how did you know the "EVA Sweet Spot" as 56 disks ina DG. Is there any logical behind it ? pls let me know.

Jeff,
Your reply has triggered my exact concern when you say " Make small vdisks of the same size, say 10gig, and use lvmstriping" . The reason behind creating this thread is to know
if size of Vdisk really matters from performance point of view. One way is to think from EVA side and another at OS-LVM side. Now EVA is quite competant to do its job without bothering OS environment ( stripping at storage level ) , then having LVM stripping is not required ( OS Overhead!!! ) . Moreover there is limitation of physical volumes supported for BC environment.

I have heard that there is RSS layout which could be taken out of EVA using some utility . It gives you the enclosure:disk
and more info .

Hence if we know association of Vdisk and Spidles , it will make the picture clear.

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Steven Clementi — Fri, 12 Aug 2005 10:23:04 GMT

The sweet spot number is based on a study performed by HP and/or a 3rd party. I do not remember WHERE I seen the number, but it has stuck with me for the past 2 years.

I will have to check some old archives for the doc that referred to it, if I even have it.

Steven

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Uwe Zessin — Fri, 12 Aug 2005 10:49:26 GMT

The data of each virtual disk is levelled across all phyiscal disk drives within a disk group (exception: VRAID-1 uses only disk pairs). Even with 240 disk drives and a 1 GigaByte virtual disk, each physical disk will store some user data.

You cannot use the RSSid/RSSindex to calculate a disk drive's position. It is possible to move a disk's position, but that will not change the RSSid/RSSindex.

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Tom O'Toole — Fri, 12 Aug 2005 13:10:57 GMT

Physical locations of drives:

I have used the following (INTEGER ARITHMETIC) formula to calculate drive locations, using the output of SSSU show disk full:

Bay = mod(Loop_ID,14)+1
Enclosure = 7*Loop_Pair - Loop_ID/14 + 1

[this would be implemented in DCL on VMS, for example, as:

$ bay=id_num-(id_num/14)*14+1
$ enc=7*pair_num-id_num/14+1

]

(If you have an expansion cabinet, it requires some modifications, but since you don't, this should work for you).

The disk location is also buried in the gui menus of CV/EVA, but that's useless for scripting unless you want to write a cheezy and complicated screen scraping program.

I suspect that how you split up your storage in terms of size/number of vdisks
doesn't matter very much, the critical point being: as long as you have enough vdisks to balance data transfer across all paths to the controllers. As others have mentioned, the data is striped over all disk in the group, regardless of the vdisk size.

For a relatively minimal outlay, you can fill your existing 2C6D config with 28 more drives, which will certainly improve performance.

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Bill Costigan — Fri, 12 Aug 2005 16:21:17 GMT

The recommendation about small vdisks and vg striping does not directly relate to eva perfomance but the way the host system generates I/Os. The theory is that if a block is spread across 4 vdisks (using LVM striping) the host will issue 4 differnet I/Os at the same time to write a block of data. The more I/Os the EVA has to work on at the same time the more I/O it can schedule.

I'm not quite sure if there is that much of a benefit. I think the eva may undo what is intended. The eva spreads the vdisk across all the disks in the DG, Therefore sequencial blocks written to a single vdisk will go to differnet physical drives. If the host system breaks up the blocks and writes to say 4 different vdisks, the blocks for each vdisk may end up on the same physical disk.

There is also a drawback with LVM stripping in that you need to always expand an LVOL with the same number of PVs. Consider an Lvol striped over 6 PVs. You run out of room, and need to add more PVs and then extend the LVOL. you cannot add 2 more PV because the lvol needs to continue to be striped over 6 PVs, so you need to add PVs in groups of 6.

I prefer, extent based striping with maybe 2 vdisk per LVOL, that way I can balance across controllers, but I don't get burned with having to keep the number of disks in a stripe set the same.

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Peter Mattei — Sun, 14 Aug 2005 10:36:17 GMT

I suggest you have a closer look at the EVA best practice whitepaper on:

http://h200006.www2.hp.com/bc/docs/support/SupportManual/lpg29448/lpg29448.pdf

Among other stuff it states that the EVA5000 scales performance wise up to 168 drives no matter if you configure one or multiple Disk Groups.
Having one DG spreads all LUN/VDISK across all spindles thus allowing all LUN to reach max performance.

Cheers
Peter

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

rccmum — Mon, 15 Aug 2005 12:24:13 GMT

Guys,

I do appretiate all your replies here.

So far I have understand that...
Vdisk - Virual Disk is logical structure/concept. Being virtual , a Vdisk is not limited to a group of physical Disk or spindles like DG does. Hence when we create a Vdisk , a space would be allocated from bunch of disks virtually. Now when you actually start using Vdisk space, EVA will always try to put data across all spindles it can have in a DG using the fixed Vdisk chunk size.
Thats why they say having 1 DG is always recommended for good performance so that there are many spindles available to EVA
( EVA is hungry for Disk spindles!!!! )

Uwe has said it rightly...

Now with all above , I still wonder about "sweet spot" ( 56 disks ) and "bad spot" ( 168 Disks ) for EVA (both marked by
HP) ???

Peter , can you comment on this pls?

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Bill Costigan — Mon, 15 Aug 2005 12:36:46 GMT

Just one minor point...

A vdisk is created in a disk group and therefore the number of spindles the vdisk spans is the same number of spindles in the disk group. (A vdisk cannot span disk groups) So creating a big disk group also leads to giving all the vdisks created in it lots of spindles to use,

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Uwe Zessin — Mon, 15 Aug 2005 14:05:56 GMT

Isn't it interesting that 'sweet spot' #1 (56 disks) is the maximum disk number supported by EVA3000/4000 and #2 (168 disks) is an EVA 5000/8000 2C12D that still fits into one cabinet?

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

Ivan Ferreira — Tue, 16 Aug 2005 08:30:08 GMT

As far i know the RSS is used for parity calculations.

Redundant Storage Sets are subgroupings of drives, usually from 6 to 11, within a disk group to provide failure separation.

For example, if a disk group has two RSSs, a drive can fail in both RSSs and we will till have VRAID5 and VRAID1 availability.

So, theorically, if you have luck enough, you can have two disks lost in a VRAID5, if the failed disks are in a different RSS.

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

rccmum — Tue, 16 Aug 2005 09:30:04 GMT

Thanks Everyone for your time replying to the thread, appretiated...

Re: Vdisk Size and Disk Spindles underneath on EVA 5000

rccmum — Fri, 19 May 2006 17:47:37 GMT

Thanks