Lvcreate, stripes and stripe size

[Bob Ferro]

My question is about the lvcreate -i and -I options. I have a VA7410 (HPAutoRAID) and I created 64 LUNs at 20G each. The VA is Fibre Channeled to a Dual Port 4G HBA. Each LUN is created with alternating RGs. I have created 4 VGs with 16 LUNs per VG. The FS and Oracle block size is 8K. My question is when creating the LVOLs, what should the number of stripes and stripe size be? Sould the number of stripes be 2 for the number of paths (8 and 8) or sould it be maybe 8 or 16? And what is a good formula to figure the stripe size?

[Tim Nelson · Accepted solution]

The number of stripes as indicated with -i is the number of disks you would like to have in the stripe.

i.e. 64 as you have stated.

lvmstriping is not that smart.

BTW, what about data protection are you planning on mirring ? If so you cannot do this with standard striping only distributed extent stripe.

As far as stripe size. I believe matching your database block size is best. There are a number of threads here on ITRC that may also state muiltples work well for specific types of databases.

[TTr]

I agree with Tim that lvm stripping with disk arrays is not a good idea. LVM striping does help with JBODs but it helps very little with LUNs on a disk array. In fact in many cases it is counterproductive.

I also think that you created too many LUNs.

If you have the time do some testing. Create a VG/LVOL without striping (over one LUN), then another with 2 stripes (2 LUNs) and so on. The run some read/write tests anc check for yourself.

As Bill Hassell has said repeatedly, "Let the hardware do the work" (the hardware in this case being the VA).

[Bob Ferro]

Thanks for the replies. I have 64 LUNs but divided among 6 VGs (16 each). I created the LVs without striping but with the BBR option disabled. I guess the only time you would use LVM striping if it is not supported on the hardware. If I did use LVM striping, what are the problems? Performance?

[Bob Ferro]

P.S. I configured the VA7410 with HPAutoRaid (Raid 0+1, Raid 5). I would have set it up with Raid 0+1 only but I was limited with the amount of disk space.

[TTr]

> If I did use LVM striping, what are the problems? Performance?

You could see a CPU/performance overhead. As the data was read from the array in stripes, all the stripes would have to be combined back into one chunk of usable data. This processing would be at the LVM layer and would take away CPU cycles and would add a delay to the I/O.

That overhead probably would not be significant but the point is you already have the striping structure at the array layer in the autoraid setup and this very same process is already taking place at the array level. Moreover, each stripe I/O that would come from different LUNs, would probably come from the same set of disk drives in the back end unless you took care to use LUNs from separate disk groups. Then there is the VA caching which helps with the back end disk array I/O.

The other issue would be maintenance. Dealing with stripes is a little bit more involved than dealing with straight LVOL setups.

Now for the size of the LUNs. If each of your LVOLs uses more than one LUN, then you should have made your LUNs bigger. But if your LVOLs are smaller than 20GB and use only one LUN then the LUN size is OK.

I typically create two or four LVOLs in each LUN to stagger the I/O at the LVOL level and keep the LUN busy at all times.

[Bob Ferro]

Great, thanks for the insight. That is more than I got 4 years ago from an HP SE. One last question is about the Bad Block Relocation. This should be disabled, correct?

[TTr]

That's correct, typically with disk array LUNs you don't need the Bad Block Relocation. There are many threads in this forum about it.

[Tim Nelson]

What you are talking about doing is called a "plaid"

A stripe of a stripe. There are some cases where the plaid is benificial and others where it is not. Lots of threads on this forum about it.

The negative is that the LVM stripe of the array stripe may disable cache and read-ahead performance benefits of the array. Because the array will no longer see the reads as sequential.

The concensious is to test one against the other in your environment.

As far as BBR, most arrays will manage their disks as well as any bad block mapping. If you let LVM manage the BBR as well it may loose track of the block map. e.g. the array remaps it somewhere then the LV maps it again somewhere else. Good luck finding your data again.

Most installation doc for the arrays will recommend disabling BBR in the LV.

[Bob Ferro]

Thanks for everyone's help. Greatly appreciated, I've learned some new things. Who said, "You can't teach a dog a new trick".