topic Re: lvextend: Not enough free physical extents available in Operating System - HP-UX

lvextend: Not enough free physical extents available

JoyBag_1 — Thu, 31 Jan 2008 15:04:04 GMT

Hi all,

I'm trying to extend a LV in a VG where there is more than enough PEs, but I'm getting the following error:

# lvextend -L 15000 /dev/vgs_prd4/lvdat2
lvextend: Not enough free physical extents available.
Logical volume "/dev/vgs_prd4/lvdat2" could not be extended.

I've recently added 4 LUNs to the VG and there more than enough PE's to cope with this request, even when taking mirroring into account:

#vgdisplay vgs_prd4
--- Volume groups ---
VG Name /dev/vgs_prd4
VG Write Access read/write
VG Status available, exclusive
Max LV 255
Cur LV 2
Open LV 2
Max PV 16
Cur PV 10
Act PV 10
Max PE per PV 3473
VGDA 20
PE Size (Mbytes) 4
Total PE 34720
Alloc PE 20500
Free PE 14220
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

#vgdisplay -v vgs_prd4 | grep -i "Free PE"
Free PE 14220
Free PE 53
Free PE 53
Free PE 60
Free PE 53
Free PE 53
Free PE 60
Free PE 3472
Free PE 3472
Free PE 3472
Free PE 3472

LV is configured as follows:

#lvdisplay -v /dev/vgs_prd4/lvdat2 | more
--- Logical volumes ---
LV Name /dev/vgs_prd4/lvdat2
VG Name /dev/vgs_prd4
LV Permission read/write
LV Status available/syncd
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 5000
Current LE 1250
Allocated PE 2500
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation PVG-strict/distributed
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c40t6d7 417 417
/dev/dsk/c42t7d0 417 417
/dev/dsk/c42t7d1 416 416
/dev/dsk/c44t6d7 417 417
/dev/dsk/c46t7d0 417 417
/dev/dsk/c46t7d1 416 416

There is 1 other active LV configured in the VG which has the same config, albeit a larger size.

I have recently deleted to other LVs from the VG.

I'm guessing the allocation policy has something to do with this, but I'm not sure how to progress. any ideas?

Thanks in advance.

Re: lvextend: Not enough free physical extents available

Torsten. — Thu, 31 Jan 2008 15:22:29 GMT

Looks like the LV is mirrored.

Check if you have enough space on *all* involved disks!

I guess you probably have only enough space on "one side" of the mirror.

Re: lvextend: Not enough free physical extents available

Michael Steele_2 — Thu, 31 Jan 2008 15:27:42 GMT

Can you paste in a '...pvdisplay -v /dev/dsk/c40t6d7 | head -25..." on your PV's?

/dev/dsk/c40t6d7
/dev/dsk/c42t7d0
/dev/dsk/c42t7d1
/dev/dsk/c44t6d7
/dev/dsk/c46t7d0
/dev/dsk/c46t7d1

Re: lvextend: Not enough free physical extents available

JoyBag_1 — Thu, 31 Jan 2008 15:37:50 GMT

Michael, as requested:

#pvdisplay -v /dev/dsk/c40t6d7 | more
--- Physical volumes ---
PV Name /dev/dsk/c40t6d7
PV Name /dev/dsk/c42t6d7 Alternate Link
VG Name /dev/vgs_prd4
PV Status available
Allocatable yes
VGDA 2
Cur LV 2
PE Size (Mbytes) 4
Total PE 3472
Free PE 53
Allocated PE 3419
Stale PE 0
IO Timeout (Seconds) default
Autoswitch On

--- Distribution of physical volume ---
LV Name LE of LV PE for LV
/dev/vgs_prd4/lvdat1 3002 3002
/dev/vgs_prd4/lvdat2 417 417

--- Physical extents ---
PE Status LV LE
00000 current /dev/vgs_prd4/lvdat1 00000
00001 current /dev/vgs_prd4/lvdat1 00003
00002 current /dev/vgs_prd4/lvdat1 00006
00003 current /dev/vgs_prd4/lvdat1 00009

Re: lvextend: Not enough free physical extents available

Tim Nelson — Thu, 31 Jan 2008 15:41:13 GMT

Looks like your lvol is Allocation PVG-strict/distributed

You will need to turn it off or change your PVG groups, you cut this info off of the vgdisplay.

Adding more disk to a distibuted stripe typically causes the stripe to be no longer ( at least on the new disks added ).

If you wish to keep some form of distributed stripe, 2 of the 4 disk you have added will need to be in PVG1 and the other two in PVG2. All new data written will only be striped to two disks. Not really a benefit but maybe better than nothing.

Your distributed mirror looks weird, I only see 7 origional disk, this is an odd number ?

If you want all vols to be distibuted accross all now 11 disks you will have to rebuild it and restore the data.

Re: lvextend: Not enough free physical extents available

JoyBag_1 — Fri, 01 Feb 2008 08:08:57 GMT

Tim, I had a feeling it may have been something to do with that, but I wanted to see if I could find a solution that avoided turning it off, but alas, they need the extra space by the end of teh day, so it looks like I'll have too.

I'm not sure why it was setup this way (with the 7 disks), it was implemented long before I joined the organisation and I can't find any documentation that references.

I'll change the allocation poloicies and see what happens.

thanks guys.

Re: lvextend: Not enough free physical extents available

Siju Vadakkan — Sun, 03 Feb 2008 05:24:10 GMT

Please provide the following output

1. vgdisplay -v vgs_prd4 | grep -e"VG Status" -e "PV Name" -e "Free PE"
2. strings /etc/lvmpvg

Re: lvextend: Not enough free physical extents available

Sandman! — Sun, 03 Feb 2008 21:09:29 GMT

Could you post the full output of lvdisplay on the LVOLs that make up the VG i.e.

# lvdisplay -v /dev/vgs_prd4/lvdat2
# lvdisplay -v /dev/vgs_prd4/

Though lvdisplay on lvdat2 shows that it has one mirror copy yet the distribution does not show it be so.

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c40t6d7 417 417
/dev/dsk/c42t7d0 417 417
/dev/dsk/c42t7d1 416 416
/dev/dsk/c44t6d7 417 417
/dev/dsk/c46t7d0 417 417
/dev/dsk/c46t7d1 416 416

So it would help if you posted the output of "lvdisplay -v" on the two lvols and also the complete "vgdisplay -v vgs_prd4". Doesn't look like you have PVGs either according to the vgdisplay report. In any case attach the above outputs along with:

# cat /etc/lvmpvg

Re: lvextend: Not enough free physical extents available

JoyBag_1 — Wed, 06 Feb 2008 12:07:25 GMT

SIJU, as requested:

#vgdisplay -v vgs_prd4 | grep -e"VG Status" -e "PV Name" -e "Free PE"
VG Status available, exclusive
Free PE 10470
PV Name /dev/dsk/c40t6d7
PV Name /dev/dsk/c42t6d7 Alternate Link
Free PE 0
PV Name /dev/dsk/c42t7d0
PV Name /dev/dsk/c40t7d0 Alternate Link
Free PE 0
PV Name /dev/dsk/c42t7d1
PV Name /dev/dsk/c40t7d1 Alternate Link
Free PE 0
PV Name /dev/dsk/c44t6d7
PV Name /dev/dsk/c46t6d7 Alternate Link
Free PE 0
PV Name /dev/dsk/c46t7d0
PV Name /dev/dsk/c44t7d0 Alternate Link
Free PE 0
PV Name /dev/dsk/c46t7d1
PV Name /dev/dsk/c44t7d1 Alternate Link
Free PE 0
PV Name /dev/dsk/c40t2d1
PV Name /dev/dsk/c42t2d1 Alternate Link
Free PE 54
PV Name /dev/dsk/c42t2d2
PV Name /dev/dsk/c40t2d2 Alternate Link
Free PE 3472
PV Name /dev/dsk/c40t2d3
PV Name /dev/dsk/c42t2d3 Alternate Link
Free PE 3472
PV Name /dev/dsk/c42t2d4
PV Name /dev/dsk/c40t2d4 Alternate Link
Free PE 3472

#cat /etc/lvmpvg
VG /dev/vg00
PVG root_pri
/dev/dsk/c1t6d0

Sandman, I have also attached the the output of the lv and vg display commands.

It is worth noting, that since last posting I have created a 3rd temp LV to get around this problem.

the PVG issue is puzzling me as the LVs are are configured PVG scrict, but /etc/lvmpvg contains no info relating to this VG.

Re: lvextend: Not enough free physical extents available

Siju Vadakkan — Wed, 06 Feb 2008 18:54:55 GMT

It seems this VG (vgs_prd4) is a shared volume group in a cluster ( cluster VG) and the LVM PVG files is not updated on the node from where ur trying to extend LV. The volume group might have created on the other node initially and the package might have switched to this node recently.

Please provide the following information

1. on the current node , run the following command

#cmviewcl

2. Please provide the following information from the other node.

#vgchange -a r vgs_prd4 # this will activate the vg in read only mode
#vgdisplay -v vgs_prd4
#lvdisplay -v /dev/vgs_prd4/lvdat2
#vgchange -a n vgs_prd4
#cat /etc/lvmpvg

We are close to the soln.

Re: lvextend: Not enough free physical extents available

JoyBag_1 — Tue, 12 Feb 2008 09:31:49 GMT

SIJU,

Sorry for the delay in replying, I've been away for the weekend.

You are correct, the VG is a Cluster VG. I can't tell you on which node teh VG was created as it was done long before I arived at the company. as for it being failed over recently, I know that I tried unsuccessfully to fail the package to the secondary node late last year, but as it failed (because oracle isn;t configured correctly), it was restarted on this node.

Anyway, here is the information you requested:

NODE1:#cmviewcl

CLUSTER STATUS
mp up

NODE STATUS STATE
node2 up running

PACKAGE STATUS STATE AUTO_RUN NODE
tstdev up running enabled node2

NODE STATUS STATE
node1 up running

PACKAGE STATUS STATE AUTO_RUN NODE
charisma up running enabled node1

See attachment for vgdisplay/lvdisplay output.

NODE2:#cat /etc/lvmpvg
VG /dev/vg00
PVG root_pri
/dev/dsk/c1t6d0

To me the lvmpvg files on both nodes seem to be missing the details for any of the VG.

Thanks Again.

Re: lvextend: Not enough free physical extents available

Siju Vadakkan — Sat, 16 Feb 2008 09:44:28 GMT

New Attachment shows there are no free PE's available.so u need to add four more disks. The below information is based on ur initial post.

If u have any query you add me google or yahoo messenger
vadakkan.siju@gmail.com
siju_vadakkan@yahoo.com

from the below information , it seems the LV is created using PVG strict distributed allocation.
#lvdisplay -v /dev/vgs_prd4/lvdat2 | more
--- Logical volumes ---
LV Name /dev/vgs_prd4/lvdat2
VG Name /dev/vgs_prd4
LV Permission read/write
LV Status available/syncd
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 5000
Current LE 1250
Allocated PE 2500
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation PVG-strict/distributed

**********************************************************
disk information collected from ur mail
Total PE 34720
Alloc PE 20500
Free PE 14220
lvextend -L 15000 /dev/vgs_prd4/lvdat2

PV Name /dev/dsk/c40t2d1
PV Name /dev/dsk/c42t2d1 Alternate Link
Free PE 54
PV Name /dev/dsk/c42t2d2
PV Name /dev/dsk/c40t2d2 Alternate Link
Free PE 3472
PV Name /dev/dsk/c40t2d3
PV Name /dev/dsk/c42t2d3 Alternate Link
Free PE 3472
PV Name /dev/dsk/c42t2d4
PV Name /dev/dsk/c40t2d4 Alternate Link
Free PE 3472
*********************************************************

1.Since the disk is mirrored using PVG Distributed allocation, you atleast four disk. Add one more disk or delete the last LV u created, I think this will free up one disk.

2.edit the /etc/lvmpvg and add the following lines at the end of file where the LV is currently nmounted.

#vi /etc/lvmpvg
VG vgs_prd4

PVG PVG-01
/dev/dsk/c42t2d2
/dev/dsk/c40t2d2
/dev/dsk/c40t2d3
/dev/dsk/c42t2d3
PVG PVG-02
/dev/dsk/c42t2d4
/dev/dsk/c40t2d4

add the 4th disk ur going to add to free up by deleting the temp LV u created just below /dev/dsk/c40t2d4 ( don't forget to add alternative path as well)

4. extend the LV

#lvextend -L 15000 /dev/vgs_prd4/lvdat2

5. Export the VG in preview mode and import it on the other cluster node

#vgexport -p -s -v -m /tmp/vgs_prd4.map vgs_prd4
# copy the above map file to the other node.

6. Logon to the other node and import the map file

#vgimport -v -s -m /tmp/vgs_prd4.map vgs_prd4.map

7. update the /etc/lvmpvg file exactly what u did on the first node

8. if u want to check the VG, just activate the Vg is ready only mode and deactivate it.

#vgchange -a r vgs_prd4
#vgdusplay -v vgs_prd
# lvdisplay /dev/vgs_prd4/lvdat2
#vgchange -a n vgs_prd4

Hope this will work.

* don;t forget to add the fourth disk.