Input on EVA 5000 Configuration

[BelaGuez]

I'd like to get some opinions from others experienced in SAN configurations.

I'm currently manageing a EVA 5000 SAN that has 3 disk shelves and 32 146GB 10k spindles. The hosts attached include;

3 Node Windows Cluster hosting 800GB+ of user file data, Print server with 200+ printers and a 2003 Exchange. All serving a ~1500 user base. Theres also a SQL server, a Oracle Host with a core application database for 50% of the user base, a VMware host plus two others.

Now my concern is the volume of I/O compaired to the number of spindles in the array. Most of these hosts are not actually live yet but very shortly will be, I'd like to know if anyone has had experience with a similar load and has opinions on the performance we're likely to get.

I need ammunition for a case to aquire alot more spindles in my opinion.

[Craig Howe_1]

Hi Samuel

When we did this, we tried to get some numbers in terms of I/O's from each of the applications that were being served up.

Eg. SQL Server - 3200 I/O's per sec

Then took an approximate of 150/160 I/O's per sec per disk.

So for the SQL example it came to about 20-21 disks.

Obviously that doesn't include growth figures.

It was pretty rough (not sure of the normal calculation methods) but seemed to work well for us. This was moving from SCSI enclosures to SAN storage.

The hard part is getting decent numbers (IO's per sec) out of the applications :)

Are you also looking at redundancy? Eg. Single/Double for drive failure?
Require a seperate disk group for logs, etc?
I only mention this as if the business requires further redundancy you may require more disks which will increase your drive count and may give you more IOs (if new disks are added to the same disk group).

Enclosure failure?
8 disk shelves and disks in sets of 8 seem to be the magic numbers I think, as per the EVA best/recommended practise guide I read a while back.

If you haven't read it let me know and I can reply again with it

Craig

[Chris Young_6]

Samuel,

My environment is as follows:

1 EVA5000, 8 shelves, 80 disks (all 146 GB).
1 Disk Group, double sparing, all VR5 volumes, about 10 TB presented.
Soon we will be adding a second 8 shelves and this will be populated with 300 GB drives in a second disk group to provide further redundancy. (data in one, transactions in the other, etc.)

5 SQL 2K Enterprise Clusters
3 W2K and 2 W2K3
3 TB total from 15 VDisks
@ 1,000,000 transactions per day.

1 Exchange 2K Cluster (W2K OS)
1 TB total from 12 VDisks
@ 500 users w/ 2 GB per day total growth

1 Clustered File Server
5 TB total from 10 VDisks

6 Misc Servers
1 TB total from 6 VDisks

1 NSR with ADIC Scalar 100 and 5 LTO1s.


My $.02? First ensure availability then worry about performance.

We are running everything VRaid5 because our existing load runs about 5% max on a 2 GB fabric. The HSV pair is just sleeping most of the time.

High Availability?
If you are running VRAID1 you may be ok, but you must ensure that each of your RSS set pairs are not in the same enclosure.

If you are running VRAID5, you really need to get 8 shelves, and then redistribute your existing RSS sets in all enclosures based on the disk RSS id.

If you don't have distributed RSS, and you loose an enclosure, you most likely will loose the entire Disk Group.

If you have any questions, don't hesitate to ask. I had to figure this out for myself after we were sold the EVA.

As far as # of spindles goes, remember that any LUN member of a Disk Group has all of it's data evenly distribued among all of the disks in the disk group.

So when I have a user hit a file, if that file is big enough it could hit all 80 disks in my disk group.

Regarding performance, did you align your partitions with 64 sectors using diskpar?

good luck,

Chris

[BelaGuez]

A bit more information, all 32 disks are in a single disk group. The majority of the LUN's are Raid 5 but there are several such as the exchange logs that are running Raid 1.

Yes the partitions (at least for all the Clustered vdisks are alligned to 64 sectors). I am aware of course of the considerations that I could make for disks and their RSS's and the shelves they're in. But with only 3 shelves (not HP best practice obviously) the RSS's mean basically nothing since with so few shelves the chances of a RSS ever working in a shelf failure situation are infintesimal.

Ideally yes I'd like to have the 8 shelves but due to various reasons the model of SAN we purchased does not contain the interconnect switches needed to go over 4 shelves. And presently the cost is too prohibative to install them.


On a side note, have you ever experienced using diskpart to extend a LUN in your cluster that has caused a failure to be able to fail over the current ownership? We extended two LUN's today (something we have done serveral times in the past) and this appeared to have no problems. Until we tried to fail the nodes over. When the other cluster node tried to mount the LUN's they were erroring out with massive checkdisk problems.

Lucky for us the system is not yet live but it was due to be very shortly and this could be a large set back for us if we can't pinpoint the problem and ensure the expandability of LUN's in future.

[Peter Mattei · Accepted solution]

You probably hit a Microsoft issue described in KB840281: Volume information is lost when you extend a partition by using the DiskPart tool and then move the volume in a Windows Server 2003 cluster.

I suggest you also have a look at KB304736: How to extend the partition of a cluster shared disk

Cheers
Peter

[BelaGuez]

Thanks Peter

I'm not a cluster expert and I didn't build it but I'll be very surprised if the guy that did knew about that second KB article as we most deffinately haven't been doing the part it mentions about shutting down all other nodes and taking the resource out of the Cluster Manager!!

[Peter Mattei]

Samuel,

I know quite a few people that do it without following all the rules defined in KB304736. Most of the time it works but for any reason sometimes not!
So if you want to be on the safe side do it!
I mean if your service is worth clustering I would not put it on risk.

Cheers
Peter

[Frank Cincotta_1]

Just to correct something in terms of not being able to go beyond four switches. Even an EVA5000 without loop switches can support more than four shelves. Their are Y type cables that get looped to support that configuration. I'm not sure HP wants to support that anymore, but we currently have one EVA 5000 with eight shelves and no loop switches. So it's definitely possible...