Trouble w/extent-based striping plus mirroring

[Dan Hardison]

I'm trying to configure extend-based striping with mirroring across 2 Fibre channel controllers using 4 seperate disks (2 on each controller). Listed below are the steps that I've taken so far. The error appears after the 'lvcreate' command. What am I doing wrong?

# vgcreate -s 2 /dev/vg02 /dev/dsk/c13t0d0 /dev/dsk/c13t1d0 /dev/dsk/c16t0d0 /dev/dsk/c16t1d0
Increased the number of physical extents per physical volume to 8683.
Volume group "/dev/vg02" has been successfully created.
Volume Group configuration for /dev/vg02 has been saved in /etc/lvmconf/vg02.con
f
# more /etc/lvmpvg
VG /dev/vg02
PVG PVG1
/dev/dsk/c13t0d0
/dev/dsk/c13t1d0
PVG PVG2
/dev/dsk/c16t0d0
/dev/dsk/c16t1d0
# lvcreate -D y -s g -m 1 -L 4096 -n lvol1 /dev/vg02
Logical volume "/dev/vg02/lvol1" has been successfully created with
character device "/dev/vg02/rlvol1".
Device file path "/dev/dsk/c16t0d0" is an alternate path
to the Physical Volume. Using Primary Link "/dev/dsk/c13t0d0".
Device file path "/dev/dsk/c16t1d0" is an alternate path
to the Physical Volume. Using Primary Link "/dev/dsk/c13t1d0".
lvcreate: Not enough free physical extents available.
Logical volume "/dev/vg02/lvol1" could not be extended.
Failure possibly caused by PVG-Strict or Distributed allocation policies.

TIA,
Dan

[Patrick Wallek]

It appears that you are trying to mirror to the same exact disks.

Basically what the message is telling you is that c13t0d0 and c16t0d0 are the SAME DISK, as are c13t1d0 and c16t1d0. You can't mirror to the same disk.

With this 2 disk set up, you could mirror between the disks (c16t0d0 mirrored to c16t1d0 with alternate links of c13t0d0 and c13t1d0) but you will NOT be able to stripe, extent based or otherwise.

If those disks are NOT the same disk, then I suggest you check your cabling.

[Ashwani Kashyap]

YOu are trying to mirror on the same disks . THe disks in PVG2 are alternate paths or PV links through the other fibre channel to the disks in PVG1 .

So essentially you have two disks only but 4 device files , two from each Fibre channel path.

So you can only mirror c13t0d0(which is the same disk reperesented by c16todo) to c13t1d0 (which is the same disk represented by c16t1d1)

[Dan Hardison]

Thanks for your replies,

c13t0d0 and c16t0d0 are indeed the same disk. I understand that I cannot mirror to the same disk but why does LVM think this?

I have a JBOD DS2405 with 12-18Gb disks being fed by 2-2Gb Fibre channel controllers. I was trying to create a VG containg 4-18Gb separate disks that would be extent-level mirrored and striped. I was following the instructions found in Doc ID 2200290240. Any assistance would be greatly appreciated.

Dan

[Patrick Wallek]

LVM thinks this because you have c13t*d0 and c16t*d0 in different PVGs. When you execute the lvcreate and specify '-s g' this is telling LVM to mirror strictly between PVGs. Well, it can't because the different PVGs are alternate paths to the same disk.

You say you have 1 DS2405 with 12 x 18GB disks. How exactly do you want your mirroring and striping set up? What do you want it to look like when you are done?

Do all 12 disks have the primary as c13t*d0 and the alternate as c16t*d0?

[Dan Hardison]

Thanks Patrick,

Your comments make perfect sense, and yes, all 12 disks have the primary as c13t*d0 and the alternate as c16t*d0.

I'd like my end result to be 3 separate VGs consisting of 4 disks each. With 1 Lvol per VG consuming all available space in said VG. The Lvols should be extent-level striped and mirrored. I also want to PVLink evey other disk in the VGs by VGReducing the primary and then VGExtedning them back.

Hope this makes sense.....
Thanks,
Dan

[Patrick Wallek · Accepted solution]

OK. I see what you to do.

Here's how you do it for 1 VG. The same procedure will work for the other 2.

Let us assume that your disks for this VG are c13t0d0, c13t1d0, c13t2d0 and c13t3d0

Do your vgcreate like this:

# vgcreate -s 2 /dev/vg02 /dev/dsk/c13t0d0 /dev/dsk/c13t1d0 /dev/dsk/c13t2d0 /dev/dsk/c13t3d0

Now, your PVGs should look like this:

VG /dev/vg02
PVG PVG1
/dev/dsk/c13t0d0
/dev/dsk/c13t1d0
PVG PVG2
/dev/dsk/c13t2d0
/dev/dsk/c13t3d0

Now do your lvcreate like this:

# lvcreate -L size_in_MB -s g -m 1 -D y /dev/vg02

This should now give an extent-based-striped VG with one mirror copy on disks c13t0d0 and c13t1d0 and the other copy on c13t2d0 and c13t3d0.

You could now use vgextend to extend your alternate links.

If you wanted some potentially better throughput you could do something like this:

Do your vgcreate like this:

# vgcreate -s 2 /dev/vg02 /dev/dsk/c13t0d0 /dev/dsk/c13t1d0 /dev/dsk/c16t2d0 /dev/dsk/c16t3d0

Now, your PVGs should look like this:

VG /dev/vg02
PVG PVG1
/dev/dsk/c13t0d0
/dev/dsk/c13t1d0
PVG PVG2
/dev/dsk/c16t2d0
/dev/dsk/c16t3d0

Now do your lvcreate like this:

# lvcreate -L size_in_MB -s g -m 1 -D y /dev/vg02

By doing it this way you have 1 PVG on 1 fibre controller and the other PVG on the other fibre controller.

Hopefully this helps!

[Dan Hardison]

Patrick,

FINALLY, the light bulb comes on!

YOU are the MAN!! Thank you very much!! Mucho pointos are coming your way. Have a great holiday.

Thanks,
Dan

[Patrick Wallek]

Dan,

Glad to be of service. That's the whole purpose of the Forums.

It's always a joy when I can help someone understand something they might not have before.

Happy holidays to you and yours as well.


P.S. Zero points for this one please.

[James R. Ferguson]

Hi Dan:

One way to leverage a physical volume group definition is with 'lvextend' since it allows you to specify the physical volume group names as 'pvg_name' arguments. Notice that 'lvcreate' doesn't offer the ability to utilize 'pvg_name' directly. To circumvent this, 'lvcreate' a zero-size logical volume and *then* use 'lvextend' with the 'pvg_name' argument. See the man pages for 'lvcreate' and 'lvextend' for more information.

Regards!

...JRF...

[Dan Hardison]

Good tip James!

Thanks,
Dan