Striping & Mirror/UX can they work together?

[Bert Woods]

Hi,
I have 4 controllers with 3 dries each. I would like to strip half of the drives and then mirror them.

I though I could figure this out myself but I'm getting lost in the "concepts(?)"

I want to control which drives are used and configure it something like this:
where p is primary drive and m is mirror
The lines denote controllers

p|m
p|m
p|m
---
p|m
p|m
p|m

so, what I would windup with is the 6 (p)rimary drives striped and then (m)irror the other 6 drives.

or is this how it's supposed to work?:

p1|m1 - vg1
p2|m2 - vg2
p3|m3 - vg3
-----------
p4|m4 - vg4
p5|m5 - vg5
p6|m6 - vg6

and then I strip vg1-6 ?

Many, many thanks

[Bert Woods]

oops,
That's "3 drives each" and "stripe half of them"
-bjw

[S.K. Chan · Accepted solution]

Not quite that straight forward. And you can't strip across different VG. There is something called "pseudo striping" or extent-based striping which can be achieved with striping/mirroring. Check this document for details.
DOCID=KBRC00011002
http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000065180752
Hope it helps ..

[James R. Ferguson]

Hi:

True striping and mirroring do not coexist. However, you *can* do 'extent-based-mirror-strips'. In this case the stripe size is equal to one physical disk extent. See the 'lvcreate' man pages for more details.

The easy way to control which physical disk are used for mirroring is with the 'etc/lvmpvg' file. See 'man 4 lvmpvg' for more details there. This file provides the basis for confining your allocation to a particular group of disks. You simply specify the 'lvmpvg' file in the 'lvextend...pvg_name' command form thereby confining your allocation (and mirrors) to the physical volumes of your choice.

Regards!

...JRF...

[John Palmer]

Hi,

You can only use 'extent based striping' if you want to mirror. That is each logical volume mirror is made up of a Physical Extent (PE) from one disk then one from the next etc.

This gives you a very wide 'stripe width', normal striping allows a width of 4 - 32768Kb. Extent based stripe width is from 1Mb but will have to be larger if you use high capacity disks. It equals the -s value supplied when you create the Volume Group.

The whole striped logical volume has to be contained within a single volume group. The easiest way to create this type of LV is to create Physical Volume Groups (see man vgcreate). This creates a list of disks in a file called /etc/lvmpvg, lvcreate then uses them to create your extent striped volume (mirrored if you like) in one operation when you use the -D y -s g flags.

In your example, I'm assuming that you have the following disks:
Controller 1
/dev/dsk/c1t1d0
/dev/dsk/c1t2d0
/dev/dsk/c1t3d0
Controller 2
/dev/dsk/c2t1d0
/dev/dsk/c2t2d0
/dev/dsk/c2t3d0
Controller 3
/dev/dsk/c3t1d0
/dev/dsk/c3t2d0
/dev/dsk/c3t3d0
Controller 4
/dev/dsk/c4t1d0
/dev/dsk/c4t2d0
/dev/dsk/c4t3d0

Put the C1 and C2 disks in one PVG with:
vgextend -g grp1 /dev/dsk/c1t1d0 /dev/dsk/c2t1d0 /dev/dsk/c1t2d0 /dev/dsk/c2t2d0 /dev/dsk/c1t3d0 /dev/dsk/c2t3d0
and the other PVG with:
vgextend -g grp2 /dev/dsk/c3t1d0 /dev/dsk/c4t1d0 /dev/dsk/c3t2d0 /dev/dsk/c4t2d0 /dev/dsk/c3t3d0 /dev/dsk/c4t3d0

You can then create an extent striped/mirrored logical volume with one command...

lvcreate -n -m 1 L -D y -s g

Regards,
John

[Bert Woods]

This is not exactly the answer I was hoping for. What is HP official solution for this? Hardware Raid?

Roughly, what kind of impact would this have on an oracle db with data files between 1M and 1G in size.

I'm trying to spread out the data across as many drives as I can w/o losing fault tolerance

[John Palmer]

An extent striped/mirrored volume gives you fault tolerance due to the mirroring and tends to even out I/O over the available drives.

What were you expecting, normal striping an mirroring? I've used this on AIX and I'm not convinced that it performs any better than extent based striping. The main trick is to tune your database such that it performs as little disk I/O as possible.

What I tend to do with Oracle databases is to allocate a straight mirrored pair for online redo logs (intensive write but small amount of data), another for archived redo logs (lots of writing and more data) and then spread the tablespaces across the rest of the available disks with extent based striping. If you don't do this then you have to give a lot of thought about how to allocate different tablespaces over the available disks probably with little previous performance information to go on. This will likely result in you having to reorganise your file placement in the light of subsequent experience.

I also use raw logical volumes rather than filesystem files but you'll need to be aware of the various implications such as backups etc.

I'm not sure what HP's official line on this but I'm sure that they'd be happy to sell you an expensive array with all the bells and whistles ;-)

Regards,
John

[Ted Ellis_2]

without a RAID controller to handle the striping for you, John's layout is about as good as you will get. If you have 72 GB drives or larger, you will also have to build the volume group with a pe size of at least 2 MB, as the max number of pe's that can be mapped to a single physical disk is 65535... if you have 36 GB drives or smaller, you can set the 1MB pe. The basic impact here is that with datafiles around a couple of MB, the file will likely reside on a single PE so any updates to that datafile will happen across a single disks. You will see benefits with the larger datafiles as writes occur across multiple spindles (disks). This is at best a cheap, middle ground to getting some performance boost compared to single disks or non-striped volume groups. John's solution also ensures that you have a mirror for all of your data and that the mirror is on different disks... so a single disk failure will not result in data-loss. We do this in non-production environments and are happy (not-thrilled) with the results. Your only other option is to go with software RAID and spend some money on an array that handles the striping and RAID for you.

Ted

[Bert Woods]

HP

IT Resource Center
CA522601 - ass821

CA633487 yyaka4
HP Top tools update account:
bwoods -- 000005


HP IT resource center
bwoods@fantasyent.com, whoisbert@yahoo.com
CA633487
yyaka4

[Bert Woods]

Well, I didn't want to do that!!!

don't worry kids I changed the passwords!!!


John, Ted,

Thanks for your suggestions. I have been toying with this for a few months and hadn't reached a point where I felt good about it.
I don't have a budget for the hardware so the s/w will half to do.
John, yes I was hoping for raid 0/1, 1/0, 10 whatever it's correctly called striping with mirroring like what you would get with a h/w solution.

right now I have the datafiles striped across six drives and I have gotten some performance increase but I need the fault protection. I will take your suggestions and break out 4 drives. Two mirrored for Redo and archive.

Again, thanks for getting me going in the right direction.

Bert