Documentation on LVM striping?

[Bill Costigan]

Hi,

Is there a white paper or something that describes how LVM does -I striping?

I created an lvol of 24MB striped across 3 physical disks (physical extent size = 4)

Extent 1 -> disk1
Extent 2 -> disk2
Extent 3 -> disk3
Extent 4 -> disk1
Extent 5 -> disk2
Extent 6 -> disk3

The stripe size is 64KB (e.g. -I 64)
How is the 64KB stripes written,
Extent 1,2,3,1,2,3 or 1,2,3,4,5,6?
What happens if I extend the lvol? do I need to extent it to physical disks in groups of 3?

I'm just trying to get an understanding of how LVM handles a number of things? including 60KB and 66KB writes to 64KB stripes.

Thanks

[Mark Greene_1]

Try here:

http://www.docs.hp.com/hpux/onlinedocs/B2355-90672/00/00/22-con.html

mark

[A. Clay Stephenson]

The exact order of the striping really doesn't matter -- and because it picks up where the last i/o left off, you really can't simply say 1,2,3. The data is striped across each physical disk in 64K chunks in round-robin fashion and yes if you must extend a striped lvol then there must be sufficient room on 3 physical volumes for the extensions.

If you were striping in 64K chucks and a process used the write() system call for a 60K write then 60K would be written, the next 60K write would write 4K in that chunk and then 56K in a new chunk on the next disk in the round-robin. A 66KB write would write 64K in the first stripe, the 2K in the next stripe.

To the best of my ability to determine, vxfs filesystems write in 64K chunks regardless of block or extent size.

[James R. Ferguson]

Hi Bill:

The 64KB stripes (their size) are written across each physical member of the volume group in round-robin order. In this case, that would be "disk1, disk2, disk3, disk1, disk2, disk3, disk11...".

If you 'lvextend' the logical volume, you will need enough physical extents on each physical volume to allocate space on each of the three disks (in this example) accounting, again, for the round-robin writing that will occur. In the case where you would need to add more physical volumes, that would mean adding them in groups of three.

As far as LVM is concerned, the physical write is done in the filesystem's block size and this is going to be "chunked" into the stripe size at that level of granularity.

Regards!

...JRF...

[Bill Costigan]

Thanks for the explanations. I was hoping to find a more detailed document on the techniques LVM uses to place data on the disks.

I know that LVM maintains a physical extent map that can be displayed using lvdisplay -v. And that it can only handle 64K physical extents or the table would be too large.

Clearly, LVM couldn't maintain a table to track each 64KB stripe, so I couldn't figure out how it kept track of each stripe.

I guess if it always allocated groups of physical extents together it would work.
e.g. stripe across first 3 physical extents when they are full stripe across next 3 extents, etc.

As long as the extent size, stripe size and number of extents ( -i ) in a stripe group remains constant, LVM can do the math to figure out the physical extent and offset within it for any data block.

I was just hoping to see this spelled out in some kind of offical document.