PVG Strict / Distributed logical volume

[Kathy Shelledy]

I have a logical volume that I want to modify so that it is PVG strict with Distributed extents. I reduced the mirror off of two physical volumes and modified the logical volume with lvchange -D f. Now it is PVG strict / Partially Distributed. When I attempt to mirror to the disks in PVG2, I get an error message that there are not enough free physical extents. I have two disks located in PVG1 and two disks located in PVG2.

PVG1

/dev/dsk/c5t0d2 17504 extents (0 free)
/dev/dsk/c5t0d0 17535 extents (4 free)

PVG2

/dev/dsk/c6t0d3 17504 extents (all free)
/dev/dsk/c5t0d1 17539 extents (all free)

I suspect that the reason that I'm getting the error message is because the two disks in each PVG are not the same size, so the round robin will not be even. Any thoughts on this?

Thanks,

Kathy

[steven Burgess_2]

Hello Kathy

The theory behind PVG's is to put disks on one controller in one PVG, the other disks down another in your second PVG.
Your /etc/lvmpvg file should in theory look something like the below

VG /dev/vg01
PVG PVG01
/dev/dsk/c11t1d0
/dev/dsk/c11t2d0
/dev/dsk/c11t3d0
PVG PVG02
/dev/dsk/c10t0d0
/dev/dsk/c10t2d0
/dev/dsk/c10t3d0

When you mirror an lvol you mirror between PVG's

ie

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c11t1d0 00000 current /dev/dsk/c10t0d0 00000 current
00001 /dev/dsk/c11t1d0 00001 current /dev/dsk/c10t0d0 00001 current


You therefore need to have enough free extents on each side to be able to create your mirror

HTH

Steve

[Kathy Shelledy]

Thanks for your reply, Steve. The issue is not mirroring, as I had the logical volume on the disks in PVG1 mirrored to PVG2. I reduced the mirror, so that I could modify the logical volume to be distributed.

The logical volume originally was distributed and then we increased disk size. During this process I changed from a distributed layout to just a PVG strict layout. Due to the disk I/O of our application and system, the movement away from distributed has greatly reduced performance. All disk I/O is going down the c5 controller rather than both the c5 and c6 controllers. As this was the only change, I have attributed the performance decrease to the fact that I am no longer distributed.

Now, why is the lvextend -m 1 failing with "not enough free physical extents"?

[Duncan Edmonstone]

Kathy,

I assume you end-game here is to have a LV in your VG with an equal number of extents on each disk in the VG, with allocation alternating between PVGs and between disks in the PVGs yes?

You can't get to this LV setup from where you are now - doing the lvchange you mentioned won't enforce the distributed extent allocation retrospectively - only for new allocations - thats why you can't extend your VG, because you only hav 4 free physical extents in PVG1.

To do what you require you will need to:

1. backup your data in the LV.
2. remove the LV.
3. change your PVGs so that only disks on the same controller are in the same PVGs
4. re-create your LV including the '-D y -s g' options on the lvcreate
5. recreate your filesystem
6. restore your data

HTH

Duncan

[Bernhard Mueller · Accepted solution]

Kathy,

it should work the way you intended to do it, i.e. the "lvchange -D f" route, *if* you had equally sized disks.

As you suspect you have two larger disks (LUNs) though and for those additional extents there is no distributed allocation possible, as soon as disk #1 in PVG2 is full.

So actually only if you have Advanced online JFS, you could *possibly* reduce the filesystem size by (31 times pv extent size) and after that lvreduce the size of the LV so that all LUNs use no more extents than your smallest LUN.

Regards,
Bernhard

[Kathy Shelledy]

Duncan - Thanks for your reply, however, it is possible to do what I want to do. I performed it on a test box successfully prior to trying it on production. The kicker was that my test box had equal size disks (and thus an equal number of extents).

Bernhard - I struggled for a while with this and then saw that the LUNs where different. We thought briefly about reducing the filesystem and logical volume, but have decided to rebuild the LUNs to be the same size instead. Thank you for your validation.