VG and Number of PVs limitation

[Samir Mehta]

Hi,

I have a question about LVM. When we create a VG, by default the number of PEs per PV is the PEs of the first PV
that we add. If it is smaller, then the number of PVs in a VG is quite OK. But if we have disks like XP512 or XP256
LUSE volumes or EMC metadisks, the size of the LUN is huge. In this case, the maximum number of PVs in that
VG comes down to certain value. One way to get rid of this is increasing the PE size considerably. Is there any reason
for this limitation , any solution to overcome this situation?
thanks and best regards,
samir

[Shahul]

Hi

Ur question is not very much clear for me. Any way I will note down what I gathered.

Maximum PVs in a VG is 255. This U can set in kernel parameters. Then ur doubt is about maximux PEs per PV. In case of lun, LVM will consider the total lun as one PV only. In this case maximum PEs per PV will be the total PEs available in that lun. I hope I made it clear...

Best of luck

Shahul

[Samir Mehta]

Hi Shahul,

Let us take a PV of 9GB. Create a VG.
pvcreate
vgcreate vgxx
vgdisplay vgxx
we will see the max PVs as 255 and max PE per PV as the number of PEs in the 1st disk that you have added during VG creation.

Let us take a LUSE volume of 200GB.
pvcreate
vgcreate vgyy
This will fail. You have to derive some value for the number of max PVs and number of max PEs per PV, and then re-execute the command, like,
vgcreate -e 50000 -p 8 vgyy
I tried many a times. Hope you understand the question now.
Is that the limitation of LVM or is there any mistake that I am doing in this process while handling higher capacity disks?
samir

[James R. Ferguson]

Hi Samir:

The fundamental restriction is that the LVM data structures must fit into one physical disk extent. The size of this table which tracks physical extents is determined by the parameters 'pe_size', 'max_lv' and 'max_pv'.

As you have already found, adjusting these in various ways can allow creation of large volume groups. The issue, of course, is that once a volume group is created, it is impossible to change the fundamental parameters if/as it is necessary to add more pnysical disk.

The limitations described, and the defaults used (pe_size=4,max_lv=255,max_pv=16) are simply "old" values which were quite valid when physical disk sizes were much smaller than today.

Regards!

...JRF...

[Samir Mehta]

Hi Shahul,

There is no kernel parameter for changing the max-pv value. Its there in the vgcreate command. Once created we can not change. And coming to the LUSE volume as one PV, I do agree that LUSE is just one disk in HP-UX or any other OS though it has many LDEVs. But the point here is the size. The size matters to decide the number of max PVs you can have in the VG

samir

[Varghese Mathew]

Hi Sameer,

We had a discussion on this subject in the recent past..

Check the like out :
http://forums.itrc.hp.com/cm/QuestionAnswer/1,1150,0xf4765220af9bd5118ff10090279cd0f9,00.html

Cheers !!!
Mathew

[Carlos Fernandez Riera]

Maybe the solution is increase PE size.

[Wodisch]

Hello Samir,

like the previous reply told you:
the ONLY real solution is to increase the
PE size dramatically!

In today's business we do have PV sizes in
the range of 10 - 100 GB each. And you will
not be able to get a substitute for the small
ones next year!

Hence you will have to think about:
- how big will the biggest PV be, which you will ever try to "vgextend" into the VG?
- how big will the smallest PV be, which you will be able to get next year or in two years?
- what restrictions on LV size will you have?

And the answers might be:
- 100GB
- 30GB
- LV will be 1GB - 1000GB in size

So, make your "vgcreate" commands like this:

vgcreate /dev/vgBIG -p 128 -l 32 -s 64 -e 1024 /dev/dsk...

That will create a VG of up to 128 PVs, each one up to 64GB in size, and up to 32 LVs.

HTH,
Wodisch

[Deshpande Prashant]

Hi Samir
The way to overcome the restriction in a existing VG for new disk capacity, will be to create a VG with higher "Max PE per PV" and higher "Max PV" numbers.
These parameters are changable only at creation of VG.

In my machine I have "Max PE per PV" changed to "10680" and "Max PV" to "32".

These values you will have to decide based on your system and change while creating VG using sam or at commandline options.

Thanks.
Prashant.

[Byron Myers]

A good tid-bit of information is that the max number of PE's in a VG is (are use to be) 65535 (value stored as a short-int in HP-UX). This does NOT include PV-Links. So, for example, a VG with pe_size=4MB can hold ONE PV with size 255 GB (4MB X 65535). If you try to add a PV-Link to this PV it will fail.

Another example, A VG with pe_size=16MB. The VG can hold 65535 X 16 MB = 1008GB.

[Shahul]

Hi

I made a mistake. What U mentioned about max PV per vg is right. U will have to set while creating vg. Thanks for reminding

Best of luck
Shahul