How does LVM lay out the Logical volumes physically on disk?

[Tim D Fulford]

I want to know how LVM lays the LV on disk.
* Does it use the inside cylinder first & move to the outside, or vica versa. Or is it random? (& if so why as this will be the worst for performance).
* Are the sectors/segments whithin the disk used cylinders used in any particular order?

Generally I need to know this because we have a performance problem & I suspect using bigger disks (with same latency & seek times) will help improve performance.

Tim

[Carlos Fernandez Riera]

I suspect is from track 0 to track n. why?

If you run dd if=/dev/vgxx/rlvolxx of=/dev/null bs=1024k and in other session you run sar -d 5 100 it will show and incremental parcial times, so i guess ...

[Tom Geudens]

Hi Tim,
It depends (as always in performance issues).
I shouldn't worry about the way LVM does it's stuff (as there's not a lot you can do about it). However, don't run to big disks without thinking this thru.
Example : Oracle Database
/oracle_mountpoint1
/oracle_mountpoint2
/oracle_mountpoint3
This kind of structure would benefit from the fact that you use small disks and give each filesystem a seperate volumegroup (and disk). Unless you have a really large database you'll otherwise never use your complete disk.

The same applies for other applications. When using a large disk, you'll probably work with a "central" volumegroup that contains filesystems from many different applications. But these will have impact on each other, thus reducing the benefit of the larger disk.

It all depends,
Tom

[John Palmer]

Hi,

LVM uses physical extents which map onto the disk hardware. These will be contiguous (not random).

Whether they start at the inside edge or outside depends on the disk hardware itself and is not really worth worrying about.

Regards,
John

[Sridhar Bhaskarla]

Tim,

LVM really doesn't care how the disk is laid out. It simply maps the physical extents on a physical volume to Logical extents. The physical extents are determined once you add the disk to the volume group. Then it depends on how you create and extend the logical volume.

-Sri

[Tim D Fulford]

OK some more detail....

I have a 10krpm 18GB 100% used disk. The performance (with caching) is about 3ms. We NEED 1.5 or maybe 2ms...

Now looking at the stats for the disks I have a few choices

0 - my current disk 18GB 10krpm disk
........3ms seek, 5.2ms latency : tot 8.2ms
1 - 15krpm 18GB disk
........2ms seek, 3.6ms latency : tot 5.6ms
2 - 10krpm 35GB disk
........3ms seek, 5.2ms latency
3 - 10krpm 70GB disk
........3ms seek, 5.2ms latency

Now my disks are 100% (near enough) used so assuming caching makes same % improvement in performance
1 - should be 32% faster (1-5.6/8.2)
2 - should be 32% faster (1-(3+.5*5.2)/8.2)
3 - should be 48% faster (1-(3+.25*5.2)/8.2)

Options 1&2 are neck & neck BUT if LVM uses inside to outside cylinder usage it will be slower than I have calculated & conversely it will be faster if it is the other way 'round. Further to this the more info I put on the outside cylinders the faster the xfre time etc. Hence my question.

I'm prepared to make lots of dummy LV's if the last LVs exist on the outside edge & then lvremove the inner ones. But if I'm going to do this it must work.

One last thing, I can't change the stripe size!!! (doubling the stripe size MAY double the performance)

Tim

[Bill McNAMARA_1 · Accepted solution]

you should be able to figure this out with the pvdisplay -v and lvdisplay -v options.

It is contiguous and from the inside out. (slower to faster)

There is no placement allowed.

Later,
Bill

[Tim D Fulford]

Many thanks Bill.

I sort of can't believe this, as reading the SAME (Stripe And Mirror Everything) instructions it emphasises the importance of using the outtermost part of the disks as the benefits are disproportionately better. Ho hum....

http://forums.itrc.hp.com/cm/components/FileAttachment/0,,0x4a8a8cc5e03fd6118fff0090279cd0f9,00.pdf

Tim

[Bill McNAMARA_1]

no problem Tim,

you can kind of workaround the placement issue by creating an lvol1 before the lvol2, then removing the lvol1...

certain other manufacturers LVM integrations do allow you to place, but, really with the disk bandwith and rpms we're talking about nowadays, your real bottleneck is the bus... it's possibly not worth the bother.

Later,
Bill