lvm design

[Steve Faidley]

We are setting up a new superdome and EMC environment and I have some questions on the optimal LVM design. Our main app is Oracle 8.1.7 or 9i. We normally setup VG's with 4 or 8 PV's. This Oracle DB will have 48 PV's and the DBA is asking that they all be in one striped VG with 1meg PE's. He is refering to a paper published by Oracle stating that this is the best layout to avoid future tuning of "Hot Spots". Does anyone know of any LVM white papers, or have experience with doing this. Is there a point at which LVM performance degrades due to the number of PV's? PE size? lvol/fs size?

Thanks in advance for your comments.

[Craig Rants]

Wow, what a design, we have our EMC setup to contain the entire database. The O/S and apps which are more static are on the servers. The connection to the EMC is redundant over the 4 fibre connections to 4 logical disk on the EMC with each having alternate links so in theory each disk would be the primary link on only one fibre connection and the secondary link, then the third alternate link, and then the forth on the forth disk. Each one is setup with a 32k stripe size because that is how the EMC works and so on....

Then we have the O/S mirrored with the along with the apps being mirrored on internal or external disks.

I haven't heard of any white paper that suggests this kind of design, but Oracle has about a million recommendations for what you should do to your server so...

GL,
C

[Arockia Jegan]

PE Size won't affect the DB performance much. Also having more or less VG and PV won't make much performance changes. The main concern is how the drives are laid out.

If you split metavolumes/physical voulmes across several different disks and I/O channels that will give you a good performance.

[Sanjay_6]

Hi Steve,

Here is a white paper, old but might be useful,

http://support2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000009825655

Hope this helps.

Regds

[Tom Geudens]

Hi,
In our environment (8.1.7) we use a volumegroup for each mountpoint. So if you have (for example for a SID called olap) :
/oraolap1
/oraolap2
/oraolap3
/oraolap4
you would also have :
vgolap1
vgolap2
vgolap3
vgolap4
Some of these volumegroups are pretty large. Remember that for most databases you would benefit from having your volumegroups spread over multiple small LUNs. This is only a problem with really large databases (datawarehouse) where using 4Gb LUNs would be ridiculous (not to mention the fact that you run into LVM limits).

The problem with the EMC array (which we have as well) is that you can't "control" whether or not two small LUNs are actually one physical disk. If I were you, I would put the EMC guys that configure your array together with your DBA's and have them work it out ...

Regards,
Tom

[Duncan Edmonstone]

I agree with Tom, the physical layout of the hypervolumes in the EMC array is going to have an eeffect on this... remeber you could stripe from one EMC LUN onto the next, and those two LUNs actually be on the same disk! Whats more in some circumstances striping on an integrated cache disk array can actually reduce performance, as it confuses the caching algorithms. Seems to me like your DBA is thinking about JBOD, where thiis kind of stipe might make sense. Talk to your EMC guys, they *should* provide you with a diagram showing the physical mapping of the LUNs in the array with respect to physical disks, FAs and DAs. From this you should be able to work out a more sensible striping policy (each column in the stripe goes to a different physical disk accessed via a different FA and, in turn DA)

HTH

Duncan

[Stefan Farrelly · Accepted solution]

I have seen a whitepaper on oracle/emc and performance (how to get it) but its from oracle or emc, not from HP. I cant find my copy. Anyway, technology has move on since and you no need to really worry about strip sizes, PE sizes etc;

We run tons of EMC and we are all Oracle. The best way to get performance is;

1. Stripe, stripe, stripe. The more controllers the better. The more disks helps, but its the extra throughput from more controllers that really helps throughput.
2. The EMC's have tons of cache (ours 8GB) so worrying about stripe size, lun size etc. doesnt really matter that much anymore. It depends on your application - fast response needed from small lookups, or massive reports needing tons of data ?
3. Run EMC's Optimizer software (from a PC) - this reports which actual disks/spindles inside the EMC are being thrashed and when you run it in active mode it moves data around in the background so that all spindles peform evenly. Perfect! This removes the need or worry about striping across the same actual spindles and caning any one of them.

This is the best way to get performance from EMC for Oracle. Of course running HP's new Cache Luns on their XP's is even better - the database lives in memory and performance is consequently improved several times!

Good luck.

[James R. Ferguson]

Hi Steve:

I think the paper to which Stefan refers may be Oracle's "Optimal Storage Configuration Made Easy". It's recommendation is "SAME" -- Stipe And Mirror Everything.

http://technet.oracle.com/deploy/availability/pdf/oow2000_same.pdf

Regards!

...JRF...