LVM Question

[J Busch]

System –
N4000 with 4 CPU's
8 -GIG of ram
2 -2GIG fiber cards
2 -HP 2405 JBOD'S with 15 36GIG disks in each
Running oracle 9i


I would like to try and strip the disks in JBOD A in three disk strip sets. That would give me 5 strip sets of 3 disks with a strip size of 64K (raid 0). Then I would like to mirror the strip sets ion JBOD A to the disk in JBOD B (raid 1). I tried to do this thru SAM but you can not strip and mirror thru SAM.

My goal is to setup Raid 0+1 for the DBA's

My questions are can this be done??

How would I go about setting this up?

Thanks

[Patrick Wallek]

LVM does NOT allow true striping (using the -i and -I options to lvcreate) and mirroring.

You can use Distributed striping (-D option to lvcreate) and mirroring, but I don't think that will give you the stripe sizes that you want.

For more information on the distributed striping option have a look at the lvcreate man page.

[A. Clay Stephenson]

You are talking about extent-based striping and it can be done. Do a search on extent based striping and you should get many procedures. The problem is that extent-based striping stripes in a minumum of 1MB chunks and that is much too large to see any real performance benefits. Your 64k size is about ideal but that is not available.

[John Waller]

As Patrick mentioned you can use the -D but this is only extent based mirror stripe. If you wish to go this route then you will need to perform something similar to the following:

vgcreate -g PVG1 /dev/vg01 /dev/dsk/c0t0d0 /dev/dsk/c0t1d0 /dev/dsk/c0t2d0
vgextend -g PVG2 /dev/vg01 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0 /dev/dsk/c1t2d0

(in the above c0t* is JBOD A and c1t* is JBOD B disks)

lvcreate -D y -s g -m 1 -l /dev/vg01

[Steve Lewis]

Sounds like the S.A.M.E. old story (sorry)

Extent based striping does not give optimum throughput but it does significantly improve load balancing, especially if you alternate the controllers for the primary paths.

Modern disks typically have an 8Mb internal buffer which helps performance even with a 1Mb extent stripe - provided you have more than 6 disks in your stripe, fast HBAs and a fast spin speed. Its OK.

The other option is to purchase a full VXFS licence and learn how to configure RAID 1/0 on that.

[TwoProc]

I've done the extent based striping - and for lots of disks - I've found that you can gain some I/O with distributed striping.

However - for an Oracle database - I usually use a base set of mount points that I prefer not to mix on spindles with other stuff or each other.

Good Layout
u1) Oracle Home + app software
u2) System, Tools, User
u3) Rollback(undo)
u4) temp space
u5) data1
u6) data2
u7) indexes for data1
u8) indexes for data2
u9) redo log interleave sets 1,3,5
u10) redo log interleave sets 2,4,6
u11) archive logs
u12) report output

Minimum Layout
u1) Oracle Home + app software + report output
u2) System, User Data, Data Area
u3) Rollback(undo)
u4) Temp
u5) Index Area, Tools
u6) redo log interleave sets 1,3,5
u7) redo log interleave sets 2,4,6
u8) archive logs

What I'm saying here is that you would probably get better performance in separation using plain mirroring than with extent based striping on such a few disks.

Why? well, sure you've got 36 disks- but only 15 usable - due to mirrors.

You've only got enough room to make 3 extent based stripe sets via PVGs. So, this means you'll have to start making concessions on your own minimum disk system above - and come up with a 3 point system you can live with...

u1) Oracle Home + app software + report output System, User Data, Data Area + R ollback(undo)
u2) Archive Logs, Index Area, Tools, Redo log interleave sets 1,3,5
u3) redo log interleave sets 2,4,6, temp space

The one above is about the best I can come up with ...

So, you could use extent based stripes - mirrored - with 5 per set mirrored onto the others.

So, in your case - is it better to have 3 large extent based striped areas? Or have 12 good data separated areas?

I'd probably bet on the 12 separated areas.

If I used distributed stripes - I'd be looking for more areas that I could build at once - like the 8 I mentioned.

That being said - for everyone one of my arguments I could myself build about 17 arguments against each. :-) and so could a bunch of others out there. So, the REAL answer will tell you to put in each one and test throughput and performance for you application.

The reason for my recommendation is that I've got some test systems with 30 disks available (sound familiar?) - and have tried it in both configurations - and while we can certainly overrun both types of systems easily enough - I get less complaints from users on the systems with fully-separated disks.

Now, on systems with much more disks available - I go back to extent based stripes.

The biggest impact I could have on small systems that helped even more than trying various disk configurations was - believe or not - larger memory caches for the database in an effort to stay off of disk as much as possible.