extend vg and lv from SAN to large sizes

[ben_scott]

Sorry for the long post...

I am trying to increase the size of /u04. I think I have hit the limit of max PE of 65535. /u04 was made up of 2 peices of our SAN and I added a third peice. I was not able to access 100 percent of this third peice when extending the logical volume(lvol_u04).
I would like some guidance on accessing the last 12 percent of the SAN peice I allocated(17.9GB). I think I understand max_pv * max_pe = max disk size, but I think I am looking at something wrong and getting confused.
I wish vgmodify could enlarge PE size from current 8MB but I read the following - "the larger the extent size (this is the one parameter vgmodify cannot alter) the larger the volume group can grow". Now I am stuck at the best way to get this done. I was then thinking of using vgmodify to change max pe/pv in vgora_bkup to 65535 by setting max_pv to 5 and max_pe to 65535 to get largest disk size available, but I am not understanding this clearly. I should be able to make mount point /u04 as large as I wish, limited only by the SAN.

Please advise, points will be generously awarded, there are plenty to go around, so thanks in advance. I think this might be an easy answer for most of you.

HPUX 11.23

Below is information on /u04:
vg name /dev/vgora_bkup
Type LVM
Current LV 1
Current PV 3
PE Size (MB) 8
Total PE 67832 (529.9 GB)
Free PE 2297 (17.9 GB)
Allocated PE 65535 (511.9 GB)
Maximum LV 255
Open LV 1
Maximum PV 16
Actual PV 3
Maximum PE/PV 24319 (189.9 GB)

Disk
255/255/0/0.5 c12t0d5 LVM vgora_bkup 150.0GB

PV Name /dev/dsk/c12t0d5
VG Name /dev/vgora_bkup
Allocatable yes
VGDAs 2
Current LV 1
PE Size 8.0 MB
Total PE 19197
Free PE 2297
Allocated PE 16900

lvol_u04 vgora_bkup LVM vxfs 511.9 GB /u04

/dev/dsk/c12t0d3 le on pv=24316(189.9 GB) pe on pv=24316(189.9 GB) allocated=100%(189.9 GB) free=0.0 MB

/dev/dsk/c12t0d4 le on pv=24319(189.9 GB) pe on pv=24319(189.9 GB) allocated=100%(189.9 GB) free=0.0 MB

/dev/dsk/c12t0d5 le on pv=16900(132.0 GB) pe on pv=16900(132.0 GB) allocated=88%(149.9 GB) free=17.9 GB

[Mel Burslan]

if you need to increase the max PE per PV number, you might want to use vgmodify command. I think this is available on 11.23 and it is supported unlike most my 11.11 systems.

[ben_scott]

Thank you, I agree on vgmodify. I'm glad they added it.

I am just confused at why the 17.9GB is left over. It would not let me go above 65535 PE for lvol_u04 as I was only trying to increase /u04 by 150GB.

[James R. Ferguson]

Hi Ben:

> It would not let me go above 65535 PE

That's expected. The maximum value for the number of physical extents per physical volume ('max_pe') is 65,535 for LVM layout 1.0. With 11.31 and LVM 2.x the limit is vastly larger:

http://docs.hp.com/en/lvm-v2/L2_whitepaper_8.pdf

Regards!

...JRF...

[Steven E. Protter]

Shalom,

vgmodify if it works.

Otherwise rebuild the volume group.

I do a little trick with volume groups.

The default max physical volumes is 255. The PE allocation takes this into account.

Take that number down to 10 or 20 or even 30 with the vgcreate -p option and you get better PE allocation and more long term flexibility.

SEP

[ben_scott]

Thank you James, I have read that as well.
Thank you Steve, if you notice though, I think my Maximum PV is at 16.

I am new to this and can't figure out...
Total PE 67832 (529.9 GB)
Allocated PE 65535 (511.9 GB)
Maximum PE/PV 24319 (189.9 GB)
...,does not seem to jive.

How did I hit the 65535 limit? Is there anyway to reclaim the 17.9GB of free space on device c12t0d5?

So, does this mean that I cannot go beyond 511.9GB for /u04 without changing the PE size, which I think means moving the data and rebuilding the VG?

[James R. Ferguson]

Hi (again):

> How did I hit the 65535 limit?

You allocated 65,535 logical extents for the logical volume. [ My original answer was a bit misleading since I failed to read your question closely enough :-( ].

> Is there anyway to reclaim the 17.9GB of free space on device c12t0d5?

No, not for use by the logical volume that has reached its 65.535 extent limitation in LVM 1.0. With LVM version 2.x this increases to 33,554,432. A better summary than the one I first provided is this:

http://www.docs.hp.com/en/5992-4589/ch01s04.html#vgversions

> So, does this mean that I cannot go beyond 511.9GB for /u04 without changing the PE size, which I think means moving the data and rebuilding the VG?

Correct, since 'vgmodify' cannot alter the 'pe_size' (which is 8MB in your case) and 8*1024*1024*65535 = 549,747,425,280 ~ 549GB

Regards!

...JRF...

[ben_scott]

Thanks a lot James.

That is discouraging.

My next task is to enlarge a /u04 on the production server. Could I get the pro's opinion on how I go about doing this. The way it is now it seems like the /u04 described below is as large as it can get. I guess I could squeeze some more space by lowering the max_pv to 6 and the max_pe to 65535 right? But then that is the limit again without rebuilding the VG to like 16MB PEs again?

My hardware(prod)...

Current Volume Group settings:
Max LV 32
Max PV 8
Max PE per PV 51200
PE Size (Mbytes) 8
VGRA Size (Kbytes) 3344

VGRA space (Kbytes) on Physical Volumes with extents in use:
PV current -n
/dev/rdsk/c3t0d1 3392 8192
Summary 3392 8192

Volume Group optimized settings (no PEs renumbered):
max_pv(-p) max_pe(-e) Disk size (Mb)
6 65535 524283
7 59644 477155
8 52220 417763

Below is information on /u04
/u04
/dev/vgora_bkup/lv_u04_a
/dev/dsk/c3t0d1
/dev/dsk/c3t0d1
/dev/dsk/c5t0d1
/dev/dsk/c7t0d1
/dev/dsk/c9t0d1
le on pv 51183 (399.8 GB)
pe on pv 51183 (399.8 GB)
current LE 51183
allocated PE 51183
PE Size (MB) 8
Total PE 51193 (399.9 GB)
Free PE 10 (80.0 MB)
Allocated PE 51183 (399.8 GB)
Maximum LV 32
Open LV 1
Maximum PV 8
Actual PV 1
Maximum PE/PV 51200 (400.0 GB)
VGDAs 2
Total PVG 0

DISKS
H/W Path(s) 0/3/1/0.20.0.0.0.0.1
0/3/1/0.20.1.0.0.0.1
0/7/1/0.10.0.0.0.0.1
0/7/1/0.10.1.0.0.0.1
Device File(s) c3t0d1
c5t0d1
c7t0d1
c9t0d1

PHYSICAL VOLUME prop.
PV Name /dev/dsk/c3t0d1
/dev/dsk/c5t0d1
/dev/dsk/c7t0d1
/dev/dsk/c9t0d1
VG Name /dev/vgora_bkup
Status available
Allocatable yes
VGDAs 2
Current LV 1
PE Size 8.0 MB
Total PE 51193
Free PE 10
Allocated PE 51183
Stale PE 0