How to increase Max_PE_per_PV???

[Andrew Beal]

Hi,

I have got a 680GB disk sitting in an EMC array. I am trying to create a volume group and specifying that disk to use as the primary disk, but I get the following error.

#vgcreate vg02 /dev/dsk/c6t0d1
Warning: Max_PE_per_PV for the volume group (65535) too small for this PV (166911).
Using only 65535 PEs from this physical volume.
Increased the number of physical extents per physical volume to 65535.
vgcreate: Volume group "/dev/vg02" could not be created:
VGRA for the disk is too big for the specified parameters. Increase the
extent size or decrease max_PVs/max_LVs and try again.

2 questions...

a) how do I increase the max_pe_per_pv?? I did a man on it, and it says you can specify an interger between 1 - 65535, though mine would be bigger than that...

b) what is the VGRA???

Thanks in advance,

Andrew

[twang]

a. No, you need to recreate the vg.
Some description from an internal paper:
The maximum disk device size can be calculated by using the information from the vgdisplay command.
example: vgdisplay /dev/vg01

Maximum Disk Device Size = (Max PE per PV) x (PE Size (Mbytes))

By default, the maximum number of physical extents or PE's, or Max PE, is set to 1016. If the first disk added to the volume group exceeds 1016 extents the Max PE per PV parameter will be correctly adjusted to accommodate the physical volume's size.
example: vgcreate -e 4064 /dev/vg_name /dev/dsk/cXtXdX

Note: If the physical extent size was the default 4 megabytes, this would allow for a 16 gigabyte (4MB*4064) disk drive.

The Max PE per PV volume group setting cannot be increased on an existing volume group. The Max PE per PV setting must be set appropriately at volume group creation time.

b. Volume Group Reserved Area(VGRA)
Some description from an internal paper:
On bootable disks VGRA is calculated:
2144K VGRA starting sector number
2912K User data starting sector number

On non-bootable disks VGRA is calculated:
(2144K + PE_Size)
4096K limit

The header file bdres.h defines the starting sector number of the VGRA to be 2144 and the starting sector number of user data is at 2912. User data starts at a fixed location to allow maintenance mode boots and access of the root logical volume via the support tape.

[twang]

Add to above b:
VGRA:

Divided into three parts. The VGDA describes the volume group and all Physical (PV) and Logical Volumes (LV) in it as well as the Logical Extent (LE)-to-Physical Extent (PE) maps.

The VGSA contains the status of PV's in the group and the stale/current status of PE's in each lvol. The MCR deals with mirrored LV's and is a record of which extents of each lvol have been modified for recovery purposes.

LVM commands that affect this area are:
The vg__ commands except lvdisplay and lvmmigrate.

[Con O'Kelly]

Hi Andrew

The first thing I'd say is I'm not quite sure of the reason for a 630GB EMC disk. Your problem is that the maximum Physical Extents (PE) for a Physical Volume is 65535. By default the Vol Grp is created with a PE size of 4MB. Therefore you will only be able to use 655535 x 4MB=262140MB. It also appears the the VGRA maximum size will be exceeded by creating a Volume Grp with this disk & using the default values.

Try creating the volume group as follows which should result in a smaller VGRA & a PE size (20MB) that will allow you to use all the disk space. Note that this will only allow you to create 20 Logical Volumes and add 4 disks to this Volume Grp. You should also be aware that the biggest disk you will be able to add to the VG in the future will be 630GB.

# vgcreate -l 20 -p 4 -s 20 /dev/vg02 /dev/dsk/c6t0d1

Cheers
Con

[Michael Tully]

The first thing I would be doing is to reduce the size of the volume. Seeing it is a LUN, it has been created as a 'META' volume by someone. (a META is a group of LUN's made up to a bigger LUN. These are usually done so they are presented to a winblows box as a drive letter. If you present say a 100 8Gb LUNs to a winblows box, it would run out of drive letters.) If you don't know how, get the size reduced so at least you some form of flexibility. In reality you should have LUN sizes being 8-10 Gb in size and you can use a number of these to create your volume groups and logical volumes. Physical disks in these EMC units are generally 18,36,73,181Gb in physical size.

[Todd McDaniel_1]

I agree that you should divide your 680GB disk into multiple LUNs of 8-10GB that can be added into a VG... make sure your maxPVs are 128 or greater so that you dont run out of PVs that can be added...

At 8.9GB, you would have 76 Luns created...Then vgcreate /dev/vg02 those 76 Luns to vg02.

Who ever is your EMC specialist needs to reevaluate why/how you have a 680GB disk... that is extremely unmanagable...

Just imagine if you had to vgexport and vgimport htis thing... that would take forever to restore.

[GK_5]

The MAX_PE_PER_PV is maximum 65535. The size of each PE default is 4MB. If you have disk with capacity more than 260GB then change the size of each PE during the creation of VG. Use option "-s"
man vgcreate for more details.

[Andrew Beal]

Hi,

thanks for your responses.

The reason for the large lun is that it is a dev box, and due to the 25 lun limitation on the EMC CX400 we needed more luns for the production servers. SO for our dev oracle database, we need 680GB divided into multiple volumes... I guess my question is really, does having a larger PE size effect how fast your volume will grow is there are lots of small files on it? (oracle out of mind when considering this).

HP believe it will not, as they said it is the filesystem block size that will determine this, as opposed to the PE size...???

Reason, is i have another app which creates alot of small files, but is required to have another LUN the same size... and i dont want it to fill up to quickly due to the larger PE size...

Your thoughts?

Thanks in advance,

Andrew Beal