Striping

[Matt Walls]

We have a large Oracle 8i database running on an XP256, with 4 way 32k stripes.

we are moving to a 512, and i heard that i may want to start using extent based striping. in that case should i stick with the deafault of 4, or should i change it to 1 meg?

if extent based striping is not a good idea, i am planning on increasing the stripe size to 64k.

Thanks for any help!

Matt

[Alan Riggs]

Extent striping is basically a workaround for systems that are incapable of supporting stripes of a smaller size. It is very unlikely that you will find a real world data set in which extent striping will outperform true striping.

[Daniel Proulx]

You may want to consider the following note I found at p.216 in book "HP-UX Tuning and Performance", by Robert F. Sauers and Peter S. Weygant, Prentice Hall, ISBN 0-13-102716-6 :

"LVM striping is not recommended with the use of EMC Symmetrix disk arrays. It can actually degrade performance, because of the segmentation of the array cache into 32K sections. All I/O's to and from the Symmetrix are via the cache, and all I/Os between the cache and the disks are 32K in size. It is often better to configure logical volumes contigously on large Symmetrix logical devices, and let the Symmetrix optimize the access."

As far as I know, XP256 and XP512 are in the class of EMC Symmetrix arrays and similar cache optimization techniques might be used. Try to find information about cache optimization techniques before sripping with 64K. May be extent striping is the way to go. 4 or 1 MB extents? I don't know. It depends on the access pattern, and the size of I/O blocks. It may also depend on the caching techniques of XPxxx array. Contrary to the note above, I would not simply configure contigous logical volumes, because filesystems for instance tend to use the space at the beginning of the logical volume first, so you could end up having high disk utilization on some disks and other disks left almost idle. I faced this problem. A coarser striping to a multiple of 4 MB extents could also be suitable. Study your case carefully and challenge "fit all" or "everybody does that" configurations.

Daniel.

[Stefan Farrelly]

Very interesting reply from Daniel.

People seem to forget that IMHO the main reason for striping is nothing to do with access times but the amount of data you can transfer. If you have an lvol striped across multiple disks across say 4 SCSI controllers, against only 1 SCSI controller, then when you say want to quickly read/write 100MB the difference in transfer times is quantum (x4) for the lvol striped across 4 controllers as opposed to 1 controller. Most large systems which can afford EMC's or XP256's tend to need to move large amounts of data quickly, not small amounts of data (unless a realtime transaction system or something similar). We use striping at HP all the time for this reason.

[Ovidiu D. Raita]

The decision to use striping depends A LOT on the configuration you have and on the applications you run.

Let's suppose you have an XP256 with 2 ACPs, one DKU, configured with Raid 5 and Raid 1. One ACP takes care of the upper B4 (lets say Raid 1) and the other one of the lower B4 (Raid 5).

Any writing performed to a Raid 5 file system goes to 4 disks from the lower B4. The striping amongst the LDEV from the same array group is out of question. A stripping amongst the disks on the same B4 is questionable too (it can help if the canisters are fully loaded and you have R1&R2).

In our case there's no other Raid 5 B4 available for striping so it doesn't make sense at all.

On the other hand, for an XP256 with 4 DKUs, 4ACPs, configured as Raid 1 the 4 way striping should improve the performance. Just make sure that each PV belongs to another ACP.

Ovidiu

[Dave Wherry]

I'm not sure I agree with Ovidiu's assessment here. With an XP or EMC (an IDC) you're writing to cache, not directly to the disks. Eventually that cache will be destaged to the disks, but, it's pretty much transparent to the host. Should not affect performance.
I believe Stefan has really hit on the important point. Stripe across interfaces. I have 4 FC interfaces on my XP256 and the load is very balanced. We came up with this configuration last year when I went to Roseville and had discussions with a few of the XP engineers. This was their suggested architecture and our I/O performance has been great.
The standard wisdom has held that Raid-5 will outperform Raid-1 in reads. If your data is not in the host buffer cache or in the XP cache, it then has to go to disk. Raid-5 should perform better. Considering all writes go to cache on the XP, Raid level should not matter. It's data going down the bus with an instant acknowledgment to the host.
I agree with extent based striping across multiple interfaces.

[Paula J Frazer-Campbell]

Hi
Do you not think that with the speed of modern disks that striping is not really necessary unless writing large chunks of data back at one time?

P.S.

This is not a comment but a question.

Paula

[Dave Wherry]

Paula,
Good question, and all I can say is it depends on the environment.
I have a couple of systems working with an XP. Fast disks, 8GB of cache and striped on the interfaces. Works great, primarily because it's accessing cache most of the time, not the disks.
I've got systems using AutoRaids with 96MB of cache. Striping the disks and interfaces is crucial.
This is one of the fun parts of Sys. Admin. Figuring it out for your environment.