reclaiming space in one VG, moving to another vg

[denise_7]

Just wanted to make sure I got this right, bu querying your inputs. Right now we got a HPUX 11.0 system and this is the database hosting system.

VGa has several logical volumes. Three of those logical volumes (lv11, lv12, lv13) are being utilized 15%.

DBA wants to release some of the unused space, say 5G from each of these three logical volumes (lv11, lv12, lv13), and

move it to VGb. VGb also has a few logival volumes. But the reclaimed space from VGa is going to be moved to VGb, but added to lv20, lv21, lv22, and lv23.

These two groups are not the root volume group, but they contain LUNs from an EMC storage. I figured this would be a little tricky, because the data is spread all over the LUNs.

How do I check and reclaim space from VGa to VGb?

Do I need to save the VGa data from the three LVs, and blow away the LV and recreate the LVs? There may be annother way you can help me with?

Thank you for your help...all you experts...

[Pete Randall · Accepted solution]

The only way to do this is if you can free up a physical volume, or LUN, in this case. A physical volume cannot be shared between Volume Groups, so, unless you can free one entire LUN by re-arranging your LV(s), this isn't going to work.


Pete

[Sridhar Bhaskarla]

Hi,

You can only add space to a volume group by adding another physical volume (PV).

This means, you will need to free up all the logical volumes from a PV (disk) in VGa and move that PV to VGb. You could accomplish by either a pvmove or by extending the mirror and reducing the mirror.

For ex., following is VGa (vgxx)'s configuration.

PV1(cxtyd1): lv1, lv2, lv3 + free space
PV2(cxtyd0): lv4

If the free space in PV1 is enough to fit lv4, then you would move lv4 to PV1.

lvextend -m 1 /dev/vgxx/lv4 /dev/dsk/cxtyd1
lvreduce -m 0 /dev/vgxx/lv4 /dev/dsk/cxtyd0

Now PV2 does not have any logical volumes and it can be removed from VGa

vgreduce vgxx /dev/dsk/cxtyd0

Now you can add this PV to VGb (vgxy)

pvcreate /dev/rdsk/cxtyd0 (use -f if required)

vgextend vgxy /dev/dsk/cxtyd0

Now you can create logical volumes in VGb or extend the existing ones.

-Sri

[A. Clay Stephenson]

It doesn't work like that. A physical volume (for our purposes that is a disk or an array LUN) can be a member of at most 1 volume group. What you can do is use pvmove to move extents on one PV to another PV IN THE SAME VG. If you can move all the extents off of a PV (either thru pvmove or backing up, removing LVOL's, creating LVOL's, and restoring) then you can vgreduce that PV from the VG and assign it to another VG.

[Chris Wilshaw]

You can only move a whole disk device between volume groups, not just free space on a device

eg:

assuming that your lun relates to a device name of /dev/dsk/c0t1d0, and contains /dev/vga/lvol1, /dev/vga/lvol2 and /dev/vga/lvol3

In order to allocate space to /dev/vgb, you would need to totally free up /dev/dsk/c0t1d0 (either by removing the lv's, or by moving them to another disk device in /dev/vga)

[Fabio Ettore]

Hi,

ULVMKBRC00013032 - Moving /opt from vg00 to a new disk/volume group

Best regards,
Ettore

[denise_7]

Pete,

Yes, that is what I thought. But while doing a vgdisplay -v /dev/VGa/xxxx I see many disks list. However, I am not sure this is a LUN. This looks like disks, how do I verify these are actual disks and not LUNs?

thanks.

[Elmar P. Kolkman]

You cannot move the space to VGb unless you can free a physical volume (using LVM term for now).

But you could, of course, create a new lvol, if 15 Gb is enough, in VGa, to replace one of the lvols in VGb and then use the newly free space in VGb to extend the lvols there.
For instance:
create lv14 in VGa
move data from lv23 from VGb to VGa and replace it by mounting lv14 where lv23 was mounted
destroy lv23
spread the new free space in VGb across lv20, lv21 and lv22 like you wanted in the first place.

[Pete Randall]

Denise,

It doesn't really matter. If the "disks" are on the EMC, they are actually LUNs, but the rules remain the same. You'll have to free an entire LUN in order to assign that LUN to another VG.


Pete

[Sridhar Bhaskarla]

Denise,

From the system's perspective, there is no difference between a LUN and a disk. All the LUNs are seen as disks.

But it is different from the storage side. Say your LUN is comprised of four LDEVS of 7GBs (LDEVs are formed from the physical disks) configured on an XP disk array. Once the LUN is freed on the server side, you can split it in three more different combinations ((1x7,1x7,1x7,1x7),(1x7,3x7),(2x7,2x7)) on the storage and present them to the system. System will them see them as disks and you can reconfigure them into your volume groups.

-Sri

[Bernhard Mueller]

Denise,

as you say you see "many" dsk devices in each volume group, it is likely that the lvols are created as "striped" lvols across all LUNs/disks in the VG.

In that case you will *not* be able to free a LUN in order to remove it with vgreduce and add it in another vg using vgextend.

Check that first. lvdisplay /dev/vga/lv##

Regards,
Bernhard

[Todd McDaniel_1]

Denise,

The only way to do this effectively is to vgexport VGa... dont use the -s option. that uses a vgid header which uses all disks formerly used... you dont want to do this.

Then when you vgimport VGa back in, only specify the actual number of disks minus 1 or 2 if you have a mirror... this will allow you to move a LUN and its mirror(if you have one) that you just freed up over to the other VGb...

------------------------------------------
vgexport -m mapfile -v VGa
vgimport -m mapfile -v VGa (minus) 1 disk or pair of disks if mirrored

vgextend VGb (extra disk from above and mirror
lvextend /dev/VGb/lvol20
lvextend /dev/VGb/lvol21
lvextend /dev/VGb/lvol22
lvextend /dev/VGb/lvol23
fsadm -F vxfs /mount/point

[Todd McDaniel_1]

Denise,

Actually I forgot that you will need to backup your data...

Then after backup, you can just destroy the LVOls on your VGa and then remove a disk(with mirror) and then rebuild/lvcreate your LVOLs... but smaller this time.

This is a bit easier than vgexport/vgimport.

[denise_7]

Since VGa and VGb have many more logical volumes that what I listed, would it be posasible to take lv11, lv12, lv13 from VGa (note that there are many other lvs in this group) and

Move it to VGb? Could this be done with pvmove? After this is done, I then can extend lv20, lv21, lv22, and lv23?

Thanks for all of your inputs...

[Sridhar Bhaskarla]

Denise,

You cannot use 'pvmove' between two volume groups.

Only way is to move the PVs from VG to another. To move the PVs, you will need to free up logical volumes on it.

-Sri

[Marco Santerre]

Unfortunately, no..

pvmove works only within the same VG.. so unless you have space on another LUN inside VGa to move lv11, lv12, lv13, to free up a complete LUN.. you won't be able to do it this way.

If you want to move lv11, lv12, lv13 to another VG, you'll have to somehow make a copy of it in VGb (either cpio, dd, or whatever other utility) and mount it on the old lv11 mount point.. which mean.. downtime.. which, seems to me, is what you want to avoid.

[Mike Miller_8]

I hate to state the obvious but it never hurts. Create an ignite tape do a full backup if possible before you make any of the changes.

Whenever you come up with a plan for a change like this something inveriably doesn't go as planned.

Good luck.

= Mike =

[Pete Randall]

Denise,

You can create new lvols in VGb for lv11, lv12, lv13. The copy the contents from the VGa versions to the VGb versions via fbackup, cpio, tar, or whatever backup method is appropriate in your situation. The you can simply unmount VGa's lv11, lv12, lv13 and remount VGb's lv11, lv12, lv13. That would work.


Pete

[Todd McDaniel_1]

I believe her original need was to move 5GB from VGa to VGb...

Denise,

What do you really have on VGa? How hard would it be to do a export/import of VGa? You would probably need 4-6 hours at least to be safe.

Do you have any $$ for purchasing disks? That is the least headache unless you dont have the budget for that right now.

[Kenneth Platz]

This is *possible*, but will require a fair bit of effort to make it work.

I am assuming that you are using filesystems for your databasaes, and not raw devices. If you are using raw devices, then this will probably not work. This will also require you to have OnlineJFS installed.

I will assume that you have the following:

Volume groups: /dev/vga and /dev/vgb
Physical volumes: /dev/dsk/diskA and /dev/dsk/diskB
Logical volumes: /dev/vga/lvol1 and /dev/vga/lvol2, mounted on /u01 and /u02, respectively.

Now the first thing you will need to do is reduce the size of the filesystems on your /dev/vga volume group:

First, perform a lvdisplay on each logical volume to find out your LV size (in megabytes). Next, with this information, executes. Suppose you currently have 10G of space on each logical volume, and you wish to reduce to 5G, or 5120MB:

# fsadm -b $(( 5120 * 1024 )) /u01
# fsadm -b $(( 5120 * 1024 )) /u02

Next, you will need to lvreduce each logical volume:

# lvreduce -L 5120 /dev/vga/lvol1
# lvreduce -L 5120 /dev/vga/lvol2

Now you will need to free *all* of the extents on one of your physical devices. First you'll want to use "vgdisplay -v" to determine which physical device is using the fewest extents (that will make it easier). Then you will need to come up with one (or more) other physical devices on the same volume group that have enough physical extents free (total) to move them onto. Suppose you want to move all of the physical extents from diskA onto diskB:

# pvmove /dev/dsk/diskA /dev/dsk/diskB

Again, diskB will need to have enough physical extents to handle all of the extents from diskA.

Next, you can vgreduce diskA out of the volume group:

# vgreduce /dev/vga /dev/dsk/diskA

And now (just to be safe), pvcreate it again:

# pvcreate -f /dev/dsk/diskA

Now you can add it to vgb:

# vgextend /dev/vgb /dev/dsk/diskB

And from this point you can create your logical volumes on it.

The easier approach (MUCH easier) would simply be to create your logical volumes on vga (and you can now see why)