XP and LVM distributed politic

[Miguel Angel Aliaga]

Helo, I am looking for any experience about using XP with LVM ditributed politic.
Actually, we have 2 XP256 ( Raid 5 ) connected by CA and we have a bottleneck problem with the distribution of the data ( Oracle database ).
When we started with XP, HP recommened us not to use distributed politic, but now they say it could be a good idea, so I want to hear about your experiences.

I know it is not a simply question, it depends of a things, but what are you using if you are working with XP ?

Thanks all of you.

[Steven Sim Kok Leong]

Hi,

In XP, the LUNs are intended to be distributed across disks. However, the "stripe" size for each LUN is usually too large to be effective in distributing read/writes to Oracle databases, considering how small the Oracle block size (default 8 kb) as well as the oracle_multiread (usually 32 kb) is.

Using the -D distributed allocation policy at the LVM level gives you 4 Mb extents (ie. extent-based stripes). My personal opinion is not to use the distributed allocation policy (ie. extent-based striping) in LVM.

For the stripe size to be effective for Oracle databases, it must be small enough. I feel that 64 kb stripe size is a good size for an oracle block size of 8 kb and a multiread of 32 kb.

I have an Oracle database on XP256. I find that OS striping of 32kb stripe sizes at the LVM level improves I/O performance substantially for the Oracle database.

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com

[Miguel Angel Aliaga]

I agree with you, I undertand you are using -I option of LVM. In this case, the max stripe size is 32KB and we have 8KB block size ( OLTP - This is a Big transactional enviroment - ) with fb_file_multiblock_size = 32, that is max i/o = 256kB. Do you think it could be a problem ?

[Roger Baptiste]

Hi,

I have run into similar problems with a poorly configured XP256 on a oracle database.
Since different folks handled it, some VGs are configured with Distributed striping (vg of 4Mb extents) and some VG's with lvstriping.

After studying it closely, i plan to change all the VGs to internal LV striping of 32kb.
Yes, we have to use lvcreate -I -i options.
Also note that this has to be consistent throughout the system.
ANother additional way to boost the performance is to have pvlinks, splitting the load through different fibre channels.

HTH
raj

[Miguel Angel Aliaga]

Hi RajMan, have you see any differences in the change, better performance ?.
We have distributed the PV of the VG`s into diferents ACP`s and Array Groups of the XP, the only stripping we are using now is the internal of the Array Groups.
Now, we are thinking about using the distributed option of LVM or stripping of LVM.

Thanks.

[Roger Baptiste · Accepted solution]

hi,



I am yet to do it. The plan is there, but
since it's 4Tb production box, it is going to be a painful process to do even the smallest change in the setup.



That is correct, but make sure you have
the printout of the ACP/array group map , such
that the PV(lun) distribution is neatly spread out across different XP controllers. The HPxp engineer should be able to give you the layout map.

< the only stripping we are using now is the internal of the Array Groups. >

Uh oh, that doesn't seem to help, as we have learnt from our experience.



Yeah, that seems the way to go. 32k is supposed to give good performance response.
Haven't had a good experience so far with the XP arrays compared to the EMC symms.

HTH
raj