Cannot lvextend - not enough free extents - again!

[ConnieK]

HPUX 11.23
Added 10 32GB LUNS to existing VG
One LV in VG
vgdisplay output

--- Volume groups ---
VG Name /dev/vg02
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 128
Cur PV 57
Act PV 57
Max PE per PV 50000
VGDA 114
PE Size (Mbytes) 128
Total PE 15333
Alloc PE 12643
Free PE 2690
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg02/lvol1
LV Status available/syncd
LV Size (Mbytes) 1618304
Current LE 12643
Allocated PE 12643
Used PV 47

lvdisplay:
--- Logical volumes ---
LV Name /dev/vg02/lvol1
VG Name /dev/vg_vg02
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule striped
LV Size (Mbytes) 1618304
Current LE 12643
Allocated PE 12643
Stripes 47 <---- ?
Stripe Size (Kbytes) 64
Bad block on
Allocation strict
IO Timeout (Seconds) default

When I try to extend the LV by available MB, I get this error:

ohons002:/dev/rdsk# lvextend -L 1960000 /dev/vg02/lvol1
Warning: rounding up logical volume size to extent boundary at size "1960064" MB.
Warning: rounding up logical volume size to extent boundary at size "1961216" MB for striping.
lvextend: Not enough free physical extents available.
Logical volume "/dev/vg02/lvol1" could not be extended.
Failure possibly caused by strict allocation policy


What am I doing wrong? Please help!

[Berd]

ConnieK,

I think you need to use vgdisplay -v vg02 to see how your spare disk space is allocated. I guess you do not have sufficient spare disk space on enough physical volumes to satisfy the strict allocation policy for striping.

HTH

Berd

[ConnieK]

The vgdisplay is there at the top. Here it is again:
--- Volume groups ---
VG Name /dev/vg02
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 128
Cur PV 57
Act PV 57
Max PE per PV 50000
VGDA 114
PE Size (Mbytes) 128
Total PE 15333
Alloc PE 12643
Free PE 2690
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg02/lvol1
LV Status available/syncd
LV Size (Mbytes) 1618304
Current LE 12643
Allocated PE 12643
Used PV 47

Here is just one of the used PV's:
PV Name /dev/dsk/c18t9d1
PV Name /dev/dsk/c16t9d1 Alternate Link
PV Status available
Total PE 269
Free PE 0
Autoswitch On
Proactive Polling On

Here is one of the newly added PV's:
PV Name /dev/dsk/c16t6d2
PV Name /dev/dsk/c18t6d2 Alternate Link
PV Status available
Total PE 269
Free PE 269
Autoswitch On
Proactive Polling On

[ConnieK]

More info. I found out this one lvol contains a Seibel database and it was specifically set up this way - striped across all existing PV's.

What I need an answer on is how can I extend the LV? Should I not be able to add all the space added to the VG? (320GB).

Please reply soon as I am sort of under the gun.... :-)

Thanks!

[Patrick Wallek]

Since the LV is set up as a striped LV, you need disk space available on ALL 47 PVs that the LV is striped across.

If even 1 PV does not have sufficient free disk space, then your lvextend will fail.

[chris huys_4]

Hi,

Patrick means to add 320GB of diskspace on a logical volume with 47 stripes, you dont need to add 10 luns of 32GByte, but 47 luns of (320 GB /47 stripes=)6.9GByte.

Greetz,
Chris

[ConnieK]

Chris and Patrick. Thank you for your insight. Could you also give me the command structure for adding the LUNs back in? I will vgreduce the 10 out first. I just don't want to have to do this again and I am not sure of the specific way to do it. Please?

[chris huys_4 · Accepted solution]

Hi Connie,

#lvdisplay -v /dev/vg02/lvol1
The 47 first logical extents of the /dev/vg02/lvol1 should give an idea what "lun" device files were used as "primary devicefiles" of the 47 "existing luns".

"In the same way" vgextend should then also be extended with 47 "device files" of the 47 new luns. And this with the goal, to spread IO as much as possible "over" all the hba controllers. When the primary device files all are added in lvmtab for the 47 newly added luns, offcourse alternate device files will need also to get added.

adding primary devicefiles

#vgextend /dev/ /dev/dsk/ ... /dev/dsk/

adding alternate devicefiles
#vgextend /dev/ /dev/dsk/ ... /dev/dsk/


Followed then offcourse with the lvextend command lvextend -L 1960000 /dev/vg02/lvol1
.

To explain it a bit better.
f.e. with a lvol of 2 stripes. (oh yes did I mention that 47 stripes is totally over the top. ;) )

gsesrv8 # strings /etc/lvmtab
/dev/vgtest
:o}B
/dev/dsk/c4t0d1
/dev/dsk/c8t3d3

The first lun of volumegroup vgtest, is c4t0d1 and thus sends its IO through controller c4, the second lun is c8t3d3 and thus sends its IO through controller c8

NOTE: normally after the above primary device files also the alternate links should have been added in lvmtab for /dev/vgtest, so a c8t0d1 and c4t3d3, but apparantly I forget to add them..

gsesrv8 # lvcreate -i 2 -l 30 /dev/vgtest

gsesrv8 # lvdisplay -v /dev/vgtest/lvol3
--- Logical volumes ---
LV Name /dev/vgtest/lvol3
VG Name /dev/vgtest
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule striped
LV Size (Mbytes) 960
Current LE 30
Allocated PE 30
Stripes 2
Stripe Size (Kbytes) 8
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c4t0d1 15 15
/dev/dsk/c8t3d3 15 15

--- Logical extents ---
LE PV1 PE1 Status 1
00000 /dev/dsk/c4t0d1 00030 current
00001 /dev/dsk/c8t3d3 00030 current
00002 /dev/dsk/c4t0d1 00031 current
00003 /dev/dsk/c8t3d3 00031 current

>> CONCLUSION -- the adding of c4t0d1 as primary device file for the first lun in lvmtab and then c8t3d3 as primary device file for the second lun, will cause the striped volume lvol3 to effectively use c4t0d1 for the first/third/fifth/.. extent and device file c8t3d3, for its second/fourth/sixth extent, causing the IO to be spread equally over the 2 controllers

Greetz,
Chris

[ConnieK]

Chris,

I am not sure if I or you are not completely understanding.... I need to add 10 LUN's to an existing VG - which I already have. Then I need to lvextend the LV. There is only one LV in this VG. RE: lvdisplay you referenced above - here is a snippet of mine -
----- snip -------------
--- Logical volumes ---
LV Name /dev/vg02/lv01
VG Name /dev/vg02
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule striped
LV Size (Mbytes) 1618304
Current LE 12643
Allocated PE 12643
Stripes 47
Stripe Size (Kbytes) 64
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c18t7d2 269 269
/dev/dsk/c18t7d3 269 269
/dev/dsk/c18t7d4 269 269
/dev/dsk/c18t7d5 269 269

etc.... - through 47 disks

--- Logical extents ---
LE PV1 PE1 Status 1
00000 /dev/dsk/c18t7d2 00000 current
00001 /dev/dsk/c18t7d3 00000 current
00002 /dev/dsk/c18t7d4 00000 current
00003 /dev/dsk/c18t7d5 00000 current
00004 /dev/dsk/c18t7d6 00000 current
00005 /dev/dsk/c18t7d7 00000 current
00006 /dev/dsk/c18t8d0 00000 current
00007 /dev/dsk/c18t8d1 00000 current

etc.....

12629 /dev/dsk/c18t11d3 00268 current
12630 /dev/dsk/c18t11d4 00268 current
12631 /dev/dsk/c18t11d5 00268 current
12632 /dev/dsk/c18t11d6 00268 current
12633 /dev/dsk/c18t11d7 00268 current
12634 /dev/dsk/c18t12d0 00268 current
12635 /dev/dsk/c18t12d1 00268 current
12636 /dev/dsk/c18t12d2 00268 current
12637 /dev/dsk/c18t12d3 00268 current
12638 /dev/dsk/c18t12d4 00268 current
12639 /dev/dsk/c18t12d5 00268 current
12640 /dev/dsk/c18t12d6 00268 current
12641 /dev/dsk/c18t12d7 00268 current
12642 /dev/dsk/c18t13d0 00268 current
EOF
------- snip--------

I cannot rebuild the VG as there is an existing database resident on this one lv.

I thought maybe I would reduce the vg by the 10 LUN's I had added and start adding them again. I understand about vgextending over the alternate paths to the controllers; I already did that.

What I need to know is:

1. Do I need to vgextend the VG differently than #vgextend vg02 /dev/dsk/c16t2d0 /dev/dsk/c18t2d0?

2. Do I need to lvextend a different way than # lvextend -L /dev/vg02/lvol1.

The man pages on lvextend give flags that I have never used - the "-m" for mirror copies and the "-l" for logical extent number. Do I need to use these flags?

I hate being an ignoramus... but I really don't get it.

;-)

[ConnieK]

Please - can anyone out there answer this?