How to extend an LVM stripped LV... ?

[Frederic Bret]

Hi,

Imagine you have a Logical Volume stripped with LVM (not extend based and no mirror) accross 2 Physical Volumes of a Volume Group when these disks are in fact 2 RAID-5 Volumes of a SAN. My question is : How can you extend the size of the Logical Volume AND keep the stripe ?

In this SAN array (are they all the same ?), the size of a volume can't be extended. All you can do is create a new volume and let the operating system deal with it.
To keep the benefit of the 2 controllers of the SAN, I want to share the load on them. The initial configuration of the stripe if well known. Then, if I want to extend the size of the LV, the first (and only) thing to do is declare 2 new PVs (2 new SAN volumes) attached to the VG with vgextend, am I right ?
Now that I have a VG with 4 PV instead of 2, how do I use lvextend to tell it to consider the 2 new PV as the following of the 2 first ? Is it implicit ?

I think it is implicit because I didn't find any other way to do it, but perhaps I missed something...

Any idea ?

Thank in advance !

[G. Vrijhoeven]

Hi,


To extend the volume group:

1. pvcreate -f /dev/rdsk/c?t?d? (the new disks)
2. vgextend /dev/vg?? /dev/dsk/c?t?d? /dev/dsk/c?t?d?

To use the two controlles to load balance just add the disk you like to a logical volume:

lvextend -L /dev/vg??/lvol? /dev/dsk/c?t?d?

Hope this will help.

Gideon

[James R. Ferguson · Accepted solution]

Hi Frederic:

If all of the existing disk have free physical extents available in sufficient quantity to equally add new extents, then no new disk need to be added. If this is not the case, you will need to add more physical disk ('vgextend') so that the strict allocation policy is satisfied. The number of disks used for extending the volume *must* equal the
number of disks already striped across. For example, to extned your logical volume over disks cXtYdz and cAtBdC, do:

lvextend -L n /dev/vgXX/lvolX /dev/dsk/cXtYdZ /dev/dsk/cAtBdC

...where cXtYdZ and cAtBdC are either new or existing disk.

Regards!

...JRF...

[Ian Dennison_1]

Can you show us what the 'lvdisplay -v' of the logical volume looks like? (trimmed if necessary)

When you say 2 controllers, do you have 2 interface cards and 2 disks, allowing 4 possible interfaces?

To stripe to the max, I would have performed the following,...

LVSIZE=99 # (Current LV Size in extents)
NEWSIZE=199 # New LV size in extents
while (( $LVSIZE <= $NEWSIZE ))
do
LVSIZE=`expr $LVSIZE + 1`
lvextend -L \ $LVSIZE /dev/vg99/lvoln /dev/dsk/disk1
LVSIZE=`expr $LVSIZE + 1`
lvextend -L \$LVSIZE /dev/vg99/lvoln /dev/dsk/disk2
done

This wil alternatively extend the logical volume across each new Disk. There is a posting about this somewhere in the ITRC but I cannot seem to find it.

Share and enjoy. Ian
done

[Ian Dennison_1]

Ignore the backslash in my previous post in the lvextend commands; my line wrapped around so I tried to put continuation characters in.

Chairs, Ian

[Sridhar Bhaskarla]

Frederic,

Just add 2 Physical volumes to the volume group if the LV is striped across 2 PVs and run the commands lvextend and extendfs and the LVM will take care of the stripes too.

Also make sure while extending the volume groups with the PVs, the first PV is on the first controller and the second PV is on the second controller. Later add the alternate links to both of them. For ex., if the new disks are on the controllers cx and cy, then

#vgextend vg?? cxt?d? (PV1)
#vgextend vg?? cyt?d? (PV2)
#vgextend vg?? cyt?d? (alt to PV 1)
#vgextend vg?? cxt?d? (alter to PV 2)


-Sri

[Alan Riggs]

I have to ask -- why are you using LVM striping on top of hardware striping (raid 5)? While it is not guaranteed that you will see a performance degredation from "striping a stripe" it is certainly the most likely result.

Have you actually benchmarked and seen performance benefits from this configuration? My standard recommendation is to let the hardware do its job whenever possible. I use LVM mirroring only with JBOD or with hardware mirroring solutions.

[Frederic Bret]

Hi,

At first, thank you for all your answers ! Thank you particulary to James and Sridhar that seem to confirm what was the only way to do it.

In fact I'm working on our future SAN configuration. As I understand that my previous example wasn't precise enough, I'll try explicit our future configuration a bit...
Our future array (an LSI one, but it seems VA7x00 serie behave almost the same way with volumes at least) can only create new volume, not extend existing ones. That's why I prefer think now about futur array extensions...

For Alan, the point is that the array is composed by 2 active-active controllers, each of them seeing its own RAID group of disks; that's why I want to share the load amongs them by spreading IOs accross 2 physical hardware paths (including the fact that this way I don't have a Single Point Of Failure with LVM alternate paths too ...). I can do this with LVM stripe or extent distribution. I'll try extend distribution (To answer to Ian, it seems that the way you use there to stripe is extent distribution, am I right ?), but I will begin with the first method (LVM stripe) because I already have experience of it with N4000s and Autoraids 12H. The fact is that performances are almost doubled when the two hardware paths (2 SCSI controllers on the N and 2 controllers on the 12H) are used simultaneously. With the SAN array (not the HP one), the gain will be more sensible as each array controller has the charge of its own set of disk and his own physical disk fibre loop.

I begin on the array by creating 2 volumes each owned by a controller. I can say in the first time my server see 2 volumes (2 PVs /dev/dsk/CxTy1Dz1 and /dev/dsk/CxTy2Dz2) delivered by the array (I put aside the point of alternate path, the associated alternate devices beeing /dev/dsk/CxTy2Dz1 and /dev/dsk/CxTy1Dz2 ). Then I begin by creating a VG associated with this 2 PVs (the controller load sharing) and I create a LV that I stripe accross 2 PVs.

Until all the extents are not all allocated, there is no problem : data are distributed in a round-robin way accross my 2 volumes (and my 2 pairs of controllers).
When my LV is full (I let the filesystem aspect behind...), I need to extend it, so as I can't extend the size of the 2 original volumes, the only way to proceed it to create 2 new volumes (OK James, as I first create the strip with a pair of PVs) to reiterate the previous scheme. I see now 2 new PVs : /dev/dsk/CxTy1Dz3 and /dev/dsk/CxTy2Dz4. I will associate them to the VG with "vgextend VGuu /dev/dsk/CxTy1Dz3 /dev/dsk/CxTy2Dz4", and now the question is centered aroud the LV : will the LV understand that with the command "lvextend -L /dev/VGuu/lvol1 /dev/dsk/CxTy1Dz3 /dev/dsk/CxTy2Dz4" it must continue to allocate new extents with its previous ability to strip accross the 2 new PVs ? James answered that yes, and I hope so too, because I don't see any other way to do it, except perhaps with Physical Volumes Groups (I even think that solutions are not exclusive)...

Fred