Removing a PV

[Michael D. Zorn]

I've read most of the threads I could find here, and I think I understand the basic PV/LV/VG concepts and commands.

I have a system (HP-UX 10.x) that has a volume group (/vgfam) with 22 named lvols spread across 8 physical disks (2GB each [old system]).

One of them is empty and unused. (No lvols on it.)

pvdisplay for that disk shows all 508 PE free.

vgdisplay shows:

...
Cur LV 22
Open LV 22
Max PV 16
Cur PV 8
Act PV 8
...
PE size 4 MB
...
Free PE 2261
----

Two of the named LVs show
LV status available/syncd
LV size 0
Current LE 0
Allocated PE 0
Used PV 0

the "physical volumes" section at the end shows for the one I'm trying to get:

PV name /dev/dsk/c9t4d0
PV status available
Total PE 508
Free PE 508

Another is just like that (c9t14d0)

I tried to extend one of my other lvols from 552 to 800MB:

lvextend -L 800 /dev/vgfam/lvol14 /dev/dsk/c9t4d0

"... not enough free physical extents available
... probably caused by contiguous allocation policy
... probably caused by strict allocation policy"

Same if I leave off the last argument.

The empty disk in question shows up in ioscan.

This may be the problem:

I tried doing fsck (had to try something) and got a "bad super block", so it's possible the disk needs rebuilding.

OK, then I tried "pvcreate":

"already belongs to a volume group".

So now I'm stuck at: how do I get that PV out of the VG?

There doesn't seem to be a "pvremove" or "pvdelete".

[Sandman!]

Could you post the output of the below-command...

# strings /etc/lvmtab | grep c9t4d0

cheers!

[James R. Ferguson]

Hi Michael:

To eliminate the physical volume from the volume group you do 'vgreduce':

# vgreduce /dev/vgfam /dev/dsk/c9t4d0

However, since that disk is unused, removing it isn't going to help you address the 'lvextend' error. You already have enough free physical extents in the volume group, so either (a) you are violating the allocation policy; or (b) you are violating the strict allocation policy. Do a 'lvdisplay' on the *logical volume* to ascertain the allocation settings.

Regards!

...JRF...

[Michael D. Zorn]

"Could you post the output of the below-command...

# strings /etc/lvmtab | grep c9t4d0"

Here it is:

/dev/dsk/c9t4d0

[Sandman!]

Hi Michael,

Which volume group does this PV belong to? You can get that with "vgdisplay -v | grep c9t4d0". Also is this system part of a MC/SG cluster?

cheers!

[Mridul Shrivastava]

If this disk is not being used by other server if this is in the cluster, and you are very much sure that there is no data on this disk. Then you try pvcreate with -f option:
#pvcreate -f /dev/rdsk/c9t4d0
then restore the vg configuration on this.then try extending the lvol.
Other way to check the disk is try dd command on this,
#dd if=/dev/rdsk/c9t4d0 of=/dev/null bs=1024 count=1000
If this succeeds then ur disk is ok if it fails there is some issue in disk and it needs replacement.

[Vinoyee Madashery Poulo]

Hi,

Execute lvdisplay /dev/vgfam/lvol14 and check what kind of allocation policy is implemented.

[vofsky]

Hi Michael,
yes,what you should do now is check the policy with the command "lvdisplay". Normally the policy should be "strict".

[Vinoyee Madashery Poulo]

Hi,
If the lvdisplay /dev/vgfam/lvol14 shows Allocation=strict/contiguous the trying to extend the lv will result in this error message. According to your observation Total PE 508 Free PE 508, is it same with vgdisplay /dev/vgfam? If so you will get the same error though you ommit the the last argument.

[Vinoyee Madashery Poulo]

Hi,
If the lvdisplay /dev/vgfam/lvol14 shows Allocation=strict/contiguous then trying to extend the lv will through this error message. According to your observation Total PE 508 Free PE 508, is it same with vgdisplay /dev/vgfam? If so you will get the same error though you ommit the the last argument.

[shardam]

Hi Michael,

Kindly let me know the details if the logical volume(lvol14) was mirrored and it's policy allocation was set to strict, if that so you need to change the value of strict allocation to none (n). By doing this you need to issue a command of "lvchange"

e.g #lvchange -s n /dev/vgfam/lvol14

From your question if you want to remove the PV from the VG, just issue the cmd:

#vgreduce /dev/vgfam /dev/dsk/c9t4d0


Best regards,
Alfredo

[Michael D. Zorn]

All of our lvols were set up with allocation=strict/contiguous.

I don't know why - we're not using mirroring.

It looks like "strict" is mainly for mirroring, but vofsky says strict is the usual. It may be that "contiguous" is what's keeping me from extending it.

From the HP docs site, it looks like "contiguous" is better - but the data we want to have in that system is always large files with simple sequential access.

Sandman: the disk I'm trying to use (c9t4d0) is in VG /dev/vgfam.

Mridul_Shrivastava: dd shows that the disk is OK. I'll run it to completion (bs=1024) and see if I get 2GB.

Vinoyee: vgdisplay /dev/vgfam shows total PE = Free PE = 508 on c9d4t0

Alfredo: the disk isn't mirrored, and the allocation is strict.

Are there any reasons not to turn off "strict" (given that we aren't mirrored)?

I think for a first step, I'll try to just create a brand new lvol on the empty disk.

vgdisplay shows two lvols that we haven't been using:

/dev/vgfam/lvol20 (and also 22)
LV size 0
Current LE 0
Allocated PE 0
Used PV 0

(Maybe they were going to use those on the empty disk (c9t4d0), but never got around to it.

I should be able to just extend one of those.

(I'll come back and give everybody points after the next round.)

lvextend -L 600 /dev/vgfam/lvol20

I'll do that after lunch. I hate doing these things on an empty stomach.

[Sandman!]

Hi Michael,

Your lvols have strict/contiguous allocation policy. Are you using PVGs (physical volume groups)? and if that's the case could you list the contents of /etc/lvmpvg and post it here.

thanks!

[Michael D. Zorn]

Sandman: nope, no PVG.

The dd finally stopped - 2GB. I successfully did

lvextend -L 600 /dev/vgfam/lvol20

vgdisplay shows
LV size 600
Current LE 150
Allocated PE 150

lvdisplay - same thing

However - - - -

lvdisplay -v shows that the LEs went to c9t0d0 (maybe because that's the first one with free extents???), which had free PE from 326 to 507, and lvol20's PEs run from 326 to 475, so c9t0d0 now has free from 476 to 507.

OK - I can work that out later (try to put it on the empty disk c9t4d0).

extendfs complains:

extendfs /dev/vgfam/rlvol20
"/dev/vgfam/rlvol20: not an hfs file system"

OK - probably because it's empty and never been used.

Maybe I have to make a fs:

mkfs -F hfs /dev/dsk/c9t0d0... wait - that already has a FS:

pvdisplay -v /dev/dsk/c9t0d0
shows

/dev/vgfam/lvol7
/dev/vgfam/lvol8
/dev/vgfam/lvol15
/dev/vgfam/lvol20 <--- the new one

So - how do I get the new lvol20 to show up and be mountable? (to show up in /etc/mnttab?)

[vofsky]

Hi Michael,
Since the policy is "strict/contiguous",you have to issue command "lvchange -C n /dev/.../lvol..." to change policy to "strict".Then you can lvextend.
"Contiguous policy" is better but it require a contiguous PE.You must allocate the contiguous PE to LV without any gap.

[Michael D. Zorn]

vofsky: I changed the allocation to "strict". Then

lvextend -L 800 /dev/vgfam/lvol14 /dev/dsk/c9t4d0

...OK

Now lvdisplay shows

Alloc: strict
LV size 800
Current LE 200
Alloc PE 200

... and the Logical Extents include the extra 62 on c9t4d0.

But bdf shows lvol14 (/data06) at the same size as before.

Maybe I need to extend the file system:

umount /data06
extendfs -q /dev/vgfam/rlvol14

Bingo!!!

bdf shows

/lvol14 ... 800811 kbytes .......

Thanks to everyone. Now I'll organize my notes.

[Sandman!]

Michael,

Removing the contiguous allocation policy allowed you to extend lvol14? Otherwise the c9t4d0 PV would not have been included for extent allocation owing to continuity of extents required for lvol14 to be contiguous. Could you attach the output of the "vgdisplay -v vgfam" to this thread. Please attach NOT post as it would certainly be more legible to read and understand.

cheers!

[Michael D. Zorn]

Sandman:

Attached is the "vgdisplay -v /dev/vgfam"

lvol14 is the one I extended.
c9t4d0 is the one it extended onto.

The other lvol I'm still trying to create is /dev/vgfam/lvol20.

pvdisplay for c9t0d0 shows 150 PEs and LEs assigned to lvol20, but I don't know how to mount a FS to it.

The vgdisplay shows

lvol20: 600MB, current = allocated = 150

mount /dev/vgfam/lvol20 /ftp
"/dev/vgfam/lvol20: unrecognized file system"

[Sandman! · Accepted solution]

Hi Michael,

The error message you are seeing when trying to mount lvol20 is normal since you have recently added space to it, but have not yet created a filesystem on it. A logical volume that doesn't contain a file system cannot be mounted and you'll get the "unrecognized filesystem" error. To fix this do the following.

1. Create a new filesystem using the raw device file:
# newfs -F hfs /dev/vgfam/rlvol20

2. Create the mount point:
# mkdir

3. Mount the block special file to the mount point created above:
# mount /dev/vgfam/lvol20

hope it helps!

[Michael D. Zorn]

Sandman:

newfs worked. After a mount, it shows up in bdf, and in /etc/mnttab.

[Michael D. Zorn]

PS: I'll close this thread since it doesn't apply any more to "removing a PV".

[Michael D. Zorn]

Closed because my major problem was solved.