RAID 4Si Striped and Mirrored 0+1 or HP MIRRORDISK/UX best read performance config

[Rod White]

Hello to the disk config Gurus.

I have two 4Si Cards (A5856A) each with 4 channels and two HP Arrays each containing two buses with 7 disks per bus.

Summary

Controllers 2
Channels 4 in use (4 not used)
Disks 28 across 4 channels (7 per channel).

HP MirrorDisk/UX

My application is an Oracle DB. Historically I've been running RAID 0 striping across a 9 disk JBOD which has been giving satisfactorily high application performance though with no redundancy. I'm a HP sysadmin and Oracle DBA so I'm aware of filesystem cache issues, striping and tablespace segregation and distribution across disks/channels/controllers.

My purchase of the new RAID controllers and hot swap arrays is to resolve the significant impact of downtime by adding redundancy.

I want to maintain the level of read and write transaction performance and controller/disk resources are not an issue so I've decided to avoid the overhead of the RAID 3 / RAID 5 parity checking and hence have resolved to go RAID (0 + 1).

The question is, how to best achieve this with the hardware controllers and hp mirrordisk/UX ?

The criteria are READ PERFORMANCE and MANAGEMENT (failed disk replacement/rebuilds).

At the moment, failed disks require a shutdown and physical replacement (not hot swap) and then a restore from tape, typically a day downtime. A slower system for an hour or two while a disk or LV is rebuilt is a vast improvement for me.

Using the simple case of creating one large filesystem which is mirrored (I can then use the same principle to create mirrored/striped filesystems for indices,data etc).


Would it be best to

A) Large hardware RAID 0 arrays, MirrorDisk/UX

Would I create two striped (RAID 0) arrays each of 14 disks using the controller and then create single disk LV's from the resultant two PVs and mirror them with MirrorDisk/UX ?

If a disk failed, the controller has no redundancy (parity) so the PV would represent itself as failed to the OS. When a replacement disk was installed, would MirrorDisk/UX rebuild the data on the failed PV ? If it did, i assume this would take a long time since the whole array would have to be rebuilt.

B) 14 * 2 disk hardware mirrors and LVM Striping (RAID 0)

Would I create 14 arrays each of 2 disks mirrored RAID 1 using the controller which would then appear to the OS as 14 PVS. I then create an LV striped across the 14 PVS. Since the 14 PVS' appear as disks I could then easily create seperate striped LV's for data, indices etc. I assume the controller would rebuild a failed disk at the hardware level and the PV would not appear failed to the OS. This config would not use MirrorDisk/UX.

C)RAID CONTROLLER

If I use the hardware and not MirrorDisk or LVM striping....

Do I create 14 X 2 disk mirrored (RAID 1) arrays and then stripe (RAID 0) and span these into one large PV or do I create two large striped (RAID 0) and spanned arrays and then mirror (RAID 1) them into one large PV ?

The HP A5856A RAID 4Si PCI 4-Channel
Ultra2 SCSI Controller
Installation and Administration Guide suggest 14 X 2 mirrored arrays then striped and spanned.

If I use the hardware controller, can I span an array across multiple controllers or only across multiple channels of one controller ?


I'm thinking that the extra disk available to the stripe is better for performance than having a single Hot Spare. As I'm fulltime onsite I can manage a swap out when the Event Monitor emails me.

Any advice or corrections to my understanding would be appreciated.

Rod

[Steven E. Protter]

Shalom Rod,

Raid 0 has no protection against disk failure. I categorically reject any configuration that is only Raid 0.

Raid 10, which your later scenarios propose provides good fault protection and good read performance. When striping is involved, the more disks the data is on, the better the peformance will be. More disks increases the chance the heads will be where they need to be when a read request comes through.

I'm all for a hot spare, which may permit you to go for one of the Raid 0 scenarios.

Finally, hardware configurations are better than mirror/ux. Not that mirror/ux is bad, but it runs on the OS and it takes cpu cycles that can be used for other things.

Hardware based configurations will lead to better overall performance due to lighter CPU load.

SEP

[Tiziano Contorno _]

Rod,
I would let the hardware take care of the mirroring, and LVM for striping. Look at the schema I attached and tell me if anything is not clear.

In this config you stop working if an entire array goes down.

Regards.

[unixdaddy]

Rod,

I'd probably go for option A simply because

a) if you hardware mirror you obviously lose 50% of your disks so out of 28 you lose 14. You won't lose as much disk space by hardware striping
b) it alwasy seems to me that striping is always more hardware intensive than mirroring so I'd prefer to do that at the hardware level and let the OS do the mirroring

option A - i think give you more space and less work at a OS level because you simple manage 2 LUNS. You can setup your disks in the vg to have alternate links so that if you lose a controller it will still see the disk down the other channel and if you lose a disk in your hardware stripe, then when it is replaced Mirrordisk would remirror to the LUN (including the new disk).

I might add that I'd probably do a raid 5 LUN at hardware level and then mirror at OS level that way losing a disk would have no affect on a LUN, once the disk is replaced it would just get rebuilt at hardware level, but you've already ruled raid 5 out.

[Tiziano Contorno _]

Hardware mirror, a disk go bad: you replace it, no LVM work, very little time to resync the hardware mirror copy on a disk.

Hardware stripe, a disk goes bad: you lose the whole LUN at LVM level that is the whole hardware striped LUN at hardware level. Replace the disk and now you need to rebuild the stripe on hardware and to resync the stale LVs in HP-UX.

Regards.

[unixdaddy]

I guess it is a preference thing.

If you hardware stripe, then OS mirror that stripe to another disk which is also a hardware stripe then you have nothing to do when a disk fails (if you have a decent hardware product). Once you replace the disk the hardware will automatically rebuild the LUN, once the LUN is rebuilt the OS will rebuild the mirror, all whilst the system is up and running off the good mirror.

Now on the other hand if you hardware mirror and you lose a disk it isn't a problem as you have a hardware mirror, but in doing so you lose 50% of your disk space and the OS has to manage a 14 disk stripe.

You know what your needs are. I would say Hardware stripe then OS mirror for performance folowed by resilency. if you want resiliency followed by performance Hardware Mirror then OS stripe.