Creating large volume group

[dictum9]

I need to create a super large volume group, with roughly 600GB of data in it. It will consist of 10 76-GB EMC disks. Any strategy for doing so?

Is there anything better than just dumping all the disks in it and then doing lvextends on LVs? Some LVs are as large as 60GB. How about contiguous? These are EMC disks and they already have RAID 7 configuration.

[James R. Ferguson · Accepted solution]

Hi:

I think given the sophistication of the SAN, that I'd elect to create the volume group (with some growth factored into it) and simply create the necessary logical volumes without stiping (or distribution) of any kind.

Do not make your logical volumes contiguous as this will greately constain their expansion should you do so later.

By a growth factor for the logical volume, I mean that if you need 10 physical volumes initially, but might want to double that later, be sure to specify '-p 20' for the 'max_pv' since once set this cannot be changed without recreating the volume group. The same applies for the other arguments to 'vgcreate' that go into the LVM geometry --- 'max_pe', 'max_lv' and 'pe_size'. Choose appropriately.

Regards!

...JRF...

[Steven E. Protter]

Shalom etc,

Some advice.

vgcreate -p number

as in

vgcreate -p 20

A volume group only has so many PE's to spread around and the default scheme is for 255 disks.

If you will have a lot less disks, you need to change the scheme to have so much disk space in one volume group.

Since you need 600 of the 760 GB of available disk capacity, you may wish to use a hot spare to provide improved reliability for the volume group. Hopefully the underlying LUN's have some kind of striping to avoid data loss in the event of a single disk failure.

I'm not familiar with raid 7, if you explain that, I or others may be able to elaborate further.

SEP

[Victor Fridyev]

I don't know about problems with big volume groups, except physical extent size
-s pe_size Sets the number of megabytes in each physical
extent, where pe_size is expressed in units
of megabytes (MB) in the range 1 to 256.
pe_size must be equal to a power of 2 (1, 2,
4, 8, etc.). The default value for pe_size
is 4 (four megabytes).

One have to use value 16 or 32 in order to create such a big vg. Additionally you can decrease number of logical volumes (-l max_lv), if pe_size is critical for performance.
See man vgcreate

HTH

[dictum9]

There is a hot spare..

I need to make clear, that these are very large LUNS, each one is 75GB. So 10 disks makes 750GB. I doubt very much it will exceed 255 disks.

Is there still a problem, am I missing something from this?

[dictum9]

PS

My PE size now is 4.

Is that a problem, should I increase it?

[Victor Fridyev]

Hi etc,

Look at manual for vgcreate.

There are some parameters in vgcreate which influent on its building and extention, e.g:
-p max_pv Set the maximum number of physical volumes
that the volume group is allowed to contain.

Please take into account that you can not change these settings after vg creation, so try to plan the VG before you begin to use it.

HTH

[dictum9]

I think I will go with PE size of 16

[James R. Ferguson]

Hi (again):

Have a look at the Software Recovery Handbook's LVM chapter-16. The Volume Group Reserved Area (VGRA) must fit into one physical extent. While a 'vgcreate' will fail (and warn you in doing so), you can obtain a better idea of what to expect by examining some of the formula and tables in the aforementioned chapter.

Regards!

...JRF...

[James R. Ferguson]

Hi :

Oops, see:

http://www2.itrc.hp.com/service/iv/docDisplay.do?docId=prodITRC/DE_SW_UX_swrec_EN_01_E/LVM.pdf

Regards!

...JRF...

[A. Clay Stephenson]

While a pe-size of 4MiB will allow up to 252GiB per physical volume; it will also limit the maximum size of any LVOL under this VG to the same value --- and that assumes you specify 64Ki as you max_pe value. By default, vgcreate is going to use your specified pe_size and a max_pe's big enough to accomodate the largest PV listed when vgcreate is run. In your case, not only must you specify a larger pe_size but also a larger max_pe size when you run vgcreate.

Specifying a max_pe of 64Ki (-e 65536) and a pe_size of 16MiB (-s 16) will allow full utilization in any combination of LVOL's.

[Steven E. Protter]

Shalom,

A combination of a larger PE size and limiting the total number of physical volumes should do the job.

The forumulas JRF referred to are what you should look at to make the decision.

The PE size does not impact anything concerning performance. I believe A. Clay has done some actual testing in this regard with Oracle.

Congrats to ITRC's newest King, Victor, my boss.

:-)

SEP