topic Re: modular vs. cache-centric disk arrays in Disk Enclosures

modular vs. cache-centric disk arrays

Duncan Edmonstone — Tue, 12 Aug 2003 14:05:47 GMT

Could someone give me a succint definition of the difference between modular (e.g. EVA, HDS9500)and cache-centric (XP/HDS9900V) disk arrays, together with pros/cons of each?

As usual, plenty of points on offer for intelligent answers - no points for repeating what someone else just posted in a different way!

Cheers

Duncan

Re: modular vs. cache-centric disk arrays

Duncan Edmonstone — Wed, 13 Aug 2003 08:01:04 GMT

Anyone?

Re: modular vs. cache-centric disk arrays

Bernd Reize — Wed, 13 Aug 2003 13:03:03 GMT

Hi Duncan,

i think, i can tell you at least some differences , but it's only my humble opinion, of course.

Mid-Range Arrays (HDS9500, EMC Clariion..)
- config changes often need reboots
- less configurable than the highend arrays
- implement only simple redundancy
- their simple architecture achives a IO Rate near the physical limit (i.e. 30 disks ~3000 IOPS)
- mostly they are optimized for troughput instead of IOPS
- a single server can drive it to its limits

High-End Arrays (HDS9900V, EMC Symmetrix)
- almost every config change can be done online
- extremly flexible in their configuration
- multiple redundancy: the huge amount of front-end and back-end adapters makes every fault a small isolated problem - it will never "take down half the machine" like with the mid-rang arrays -> highest available reliablity
- as long as your applications can use it, they achive IO rates far above the physical limits trough their cache logic
- Heavy optimized for IOPS, troughput may be not as good.
- to really use all of its resources you have to connect many (lets say, at least 8 or 16) servers to it

I think the coice is mainly what applications you need the array for, what level of reliablity you need and how much it may cost, of course.

Re: modular vs. cache-centric disk arrays

Duncan Edmonstone — Wed, 13 Aug 2003 13:24:54 GMT

Thanks,

Perhaps I need to be a little clearer on what I am after... I understand all of what you have said, but as far as I can tell these are all implicit pros/cons of the different types of disk arrays, what I'm looking for is how these products are different 'architecturally' and how the different architectures introduce different explicit pros/cons.

Thanks,

Duncan

Re: modular vs. cache-centric disk arrays

Stuart Abramson_2 — Wed, 13 Aug 2003 14:03:29 GMT

This is "tongue-in-cheek" but true:

The EMC Symmetrix is the biggest, fastest, most reliable, most flexible, most feature-rich disk in the world. It has the best support of any vendor. It has the best, most complete software surrounding it.

If you can afford it, buy it.

If you can't afford it, buy one of the others.

Re: modular vs. cache-centric disk arrays

Peter Mattei — Wed, 13 Aug 2003 14:47:34 GMT

Stuart

Sorry to disappoint you, but the Symmetrix is neither the biggest, the fastest, the most reliable nor the one with the most features.

It's the XP series!! Look for the XP1024.

Here's why.
Size:
Sym DMX3000 84TB max (just announced) XP1024 149TB (available for ages)

Features:
- Almost the same but the XP has async remote replication with in-order delivery for years while EMC just announced it for the DMX!
- XP port control ressource manager. Feature to prioritize and/or limit IOs of individual servers. Not available on Symmetrix.
- RAID5: available on XP not on Sym. They still use proprietary and slow RAID-S
- MetroCluster solutions for Solaris, AIX, Windows and Linux are only available on XPs!
- ???

Reliability
- XP has mirrored Cache, Sym just Raid protection
- Concurrent HW and microcode maintenance is common on XPs since 1999. EMC claims online microcode upgrade possible but EMC Engineers always ask for offline upgrades!! Why?!

Security:
- Anybody can dial into the Sym and hack it. The XP does dial-back to secure, predefined numbers only.
There are many more things, but your input was definitely not what Duncan asked for ???

Peter

Re: modular vs. cache-centric disk arrays

Duncan Edmonstone — Thu, 14 Aug 2003 07:58:43 GMT

Hmmm - definately NOT what I asked for!

Religious wars can take place on someone elses thread...

Duncan

Re: modular vs. cache-centric disk arrays

Leif Halvarsson_2 — Thu, 14 Aug 2003 09:53:09 GMT

Hi,
Maybe you can find som information here:
http://www.evaluatorgroup.com/English/Services/SPC_Benchmarks.html

Re: modular vs. cache-centric disk arrays

Alzhy — Thu, 14 Aug 2003 13:14:52 GMT

Duncan,

I hope this addresses your post:

MODULAR ARRAYS:
- lower cost
- easier deployment
- Smaller Cache Sizes
- simple "caching" algorithms
- Caches can be easily overrun
- acceptable performance for OLTP, small DSS
- start small, grow as your needs.
- average "SANability"

Good Bets: HPQ (Compaq) EVA Line, Amdahl/Fujitsu, Clarion, SUN StoreEdge Mid-range Series.

CACHE-CENTRIC ARRAYS:
- Very large Caches (both Back and Front end)
- Better IO througputs - hence very good for large DSS/DW implementations as wellas OLTP.
- More Multipathing options
- More In-Box tools for replication, mirroring, etc...
- Virtually no downtime, Vendor CallBack features
- InBox OS offers intelligent caching algorithms to handle very complex IO patterns.
- Scalable
- Mutli-platform connectivity -- FICON, FC

Best Bet: Hitachi (either HP's XP line or Sun's StoreEdge 99xx line), EMC's DMX line

Re: modular vs. cache-centric disk arrays

Mike Naime — Fri, 15 Aug 2003 03:54:02 GMT

I'm looking at this from the Compaq side of the house.

=========================================
The HSG80 controller can be programmed from the CLI by a serial cable connection that can be remotely accessed through a terminal server. Or you can use GUI tools that require client software to manage once you setup your inital Units/LUNS.

The HSG80's in the BAXXX or EMAxxxx cabinets have you managing the spindles. You make your storagesets based on the physical spindles.
I currently can only make a 6x146GB raidset without breaking any of the redundancy rules.
This gives me a raidset that has 700GB of usable space as the largest raid that I can make.

If I stripe mirrors, I loose 1/2 my space in the storageset to redundancy, but I can make an even larger LUN up to 30*{disk size} as long as I stay under the 2TB controller limit.

===================================

The EVA requires the SAN appliance as the front end to manage/monitor the EVA through the supplied GUI tools.

The EVA uses as many Fibre channel disks (Up to 240) that you want to put into a disk group.

After you set your paranoia level (Sparing and redundancy) you manage the space in 1GB increments exactly how much disk space you want to present to each host.

We have 168 146GB disks in one disk group in our EVA. (2c12D + 0c12D expansion)

=====================================
We looked at the XP system. While it had the ability to expand to hold more disks than the individual HSG/EVA it had a much higher cost of ownership for us than the EVA did.

Mike Naime

Re: modular vs. cache-centric disk arrays

Vladimir Vybiral — Fri, 15 Aug 2003 07:36:40 GMT

Hello, just a short word of opinion:
Today, the cache-centric arrays offermore or less the same levels of features (remote mirroring, speed, availability, heterogeneity, you name what) as the modular arrays do.
Differences:
Cache-centric - Management and configuration is often tedious and involves sometimes vendor engineers to come to Your site
Modular - if we speak about modular example of the EVA5000, the management is EASY and done in minutes, plus the array intelligence does some things by itself (if You trust it - but I would) - like load levelling, hot sparing, cache allocations etc.

Cache-centric - The initial cost is HIGH, as You need to buy the engine, motor of the array, which is itself a box that occupies some space in Your serverroom. Need more space? Call engineers, let them add drives, init them, init containers, format, include, create disk groups, create mirrorsets and strip sets, create LUNs... You know this take time and a mistake means starting from scratch...
Modular - buy what You need today - a set of controllers and disk shelves, put them in a rack, hook and run. Need more space? Buy more disks, plug them in, and the space is automatically claimed, go to management, add the disks into a disk group of ANY size, data is automatically levelled (for more spindles = more speed), select a LUN a grow it by 100GB... (EVA example)
Need more speed? Either by just plugging disks You immediatelly increase the speed (as You have more spindles and data are striped automatically across everything) or buy another set of controllers, hook them to You rack, put some disks behind...

Cache-centric - Need mainframe connection? Go for a monolithic array. Mainframe computers are the only systems today that do not have a front-end cache so You need to have the xxxxx cache on the array. All other systems (win, UX, Linux, etc.) have their own cache so for a well tuned system You typically do not need those huge amounts of cache on the array.
Modular - typically 1-4GB of cache, which is NOT_MUCH certainly, but is okay for the delayed processing of stripe/parity write/rewrite. If You want more, buy another set of controllers :)

Life is soooo easy with the modular virtual arrays like VA or EVA. This is what I prefer unless there is a need for ESCON/FICON connection. The modulars do not support mainframe.

Oooops, wanted to write this short, but it turned out to be a bit wordy... :)

Vladimir

Re: modular vs. cache-centric disk arrays

Vincent Fleming — Fri, 15 Aug 2003 13:32:07 GMT

Re: modular vs. cache-centric disk arrays

Duncan Edmonstone — Fri, 15 Aug 2003 14:09:23 GMT

Thanks all, I think this is pretty much confirming what I already suspected... I've tried to abstract this out into a couple of diagrams - I know this is generalising a lot, bu this seems to me to be pretty much what the differences are architecturally - see what you think, and put me right if I am miles off!

Cheers

Duncan

Re: modular vs. cache-centric disk arrays

Mike Naime — Sat, 16 Aug 2003 05:42:29 GMT

For the Compaq modular stuff.

The (Modular) HSG80 and EVA can have shared (Mirrored) cache between the controllers. Your drawing shows it on each controller with no interconnect. I would place the cache in the middle between the two controllers.

The HSG80 has 6 SCSI channels/shelves with up to 14 drives per channel/shelf.

The EVA has up to 18 Fiber based SCSI channels (Shelves) of up to 14 drives per shelf/channel.

Originally they had fiber loops in teh design. Now they are using a special brocade fibre switch to interconnect all the shelves and controllers. A SAN within a SAN!

The EVA supports 240 drives, but it will hold 252 drives. I have not figured out yet which slots you are not supposed to populate! :-)

Re: modular vs. cache-centric disk arrays

Vladimir Vybiral — Mon, 18 Aug 2003 05:14:30 GMT

A short answer on the last point:
You can populate up to 240 slots in any shelve. They are a way of "autoselect". The ones that You leave out depend on You and Your config.
The back-end on EVA is still a loop, though switched. (the devices there are four Vixel loopswitches, not Brocade). The compatibility does not suffer (as it is not related to the front end, of course), and it is cheaper than Brocade. There is one "dual loop" per shelve or enclosure, switched, virtually forming two loop addressing spaces. That is why You max out at 240 drives (one loop is 127 addresses - some reserved -2 for each ctrl = 120).

BR, Vladimir

Re: modular vs. cache-centric disk arrays

Duncan Edmonstone — Mon, 18 Aug 2003 07:13:38 GMT

Guys, I think your missing the point here... I'm not trying to understand the architecture of any one disk array, but the general differences in architecture between different classes of disk array.

Thanks,

Duncan

Re: modular vs. cache-centric disk arrays

Vincent Fleming — Mon, 18 Aug 2003 09:57:51 GMT

Duncan,

IMHO, Your diagrams are essentially correct. There are variations on the theme from different vendors, but you've displayed a good understanding of the basic technology with those diagrams.

Good luck,

-Vince