vgreduce won't work, physical extents in use supposedly

[Tom Weber_4]

I???m having a problem doing a vgreduce on a LUN. Here???s what I get:

vgreduce /dev/vg01 /dev/dsk/c0t0d2

vgreduce: Physical volume "/dev/dsk/c0t0d2" could not be removed since some of its
physical extents are still in use.

Here???s the pvdisplay ???v of /dev/dsk/c0t0d2:

--- Physical volumes ---
PV Name /dev/dsk/c0t0d2
VG Name /dev/vg01
PV Status available
Allocatable yes
VGDA 2
Cur LV 0
PE Size (Mbytes) 4
Total PE 7226
Free PE 6976
Allocated PE 250
Stale PE 0
IO Timeout (Seconds) default


--- Physical extents ---
PE Status LV LE
0000 free 0000
0001 free 0000
0002 free 0000
0003 free 0000
0004 free 0000
0005 free 0000
???????????????????????????.

It looks as if there is 250 PE allocated but yet the physical extents all show as being free. I checked each of the logical volumes and they do not report using this disk nor does pvdisplay show any logical volumes assigned. This disk had been previously used and I had mirrored the lvols over to a new array. Since then, the mirrors were broken on the old array. This is on HP-UX 10.20

So how would I go about cleaning this up so I can delete this LUN? Is there a way to delete the physical extents SUPPOSEDLY in use? This is the first time I couldn???t find a specific answer by doing a search. I found some close but not quite what I need.

Thanks for your help!

[John Poff]

Hi,

I don't know how to clean it up, but I would try creating a LV of 250 PE on that disk and see if pvdisplay on that PV reports 250 PE in use or 500 PE in use. Then, delete the new LV and see what it reports. Maybe the LVM header for that PV is messed up and creating/deleting an LV on it could clean it up?

JP

[Sandip Ghosh]

The same situation happens when you delete the device file from the /dev/vg01 directory. Try to create a device and raw_device file in the /dev/vg01 directory. Then look for the vgdisplay -v /dev/vg01 and pvdisplay . Now you should find the same device. Then remove the logical volume by lvremove.

Sandip

[Tom Weber_4]

creating a new lvol of 250 PE just added to the PE allocated for a total of 500. I then did an lvremove on the newly created lvol and I'm back to where I was. Thanks for the suggestion though!

[John Poff]

Ok. You might try a 'vgcfgrestore' on that PV (if you have a good, recent copy of the lvmconf file for that VG). If that doesn't work, and you can get some downtime for the VG, you could try a vgexport/vgimport on the whole VG.

JP

[Tom Weber_4]

Sandip, all of my device files are there as they should be.

John, all of my old lvmconf files for this VG would include this disk device I'm trying to get rid of.

I just want to be able to vgreduce this device and then delete the LUN.

Any other ideas out there?

Thanks.

[Sandip Ghosh]

What about your lvmtab file. Is the disk is over there? If it is still over there, rename the lvmtab file and do a vgscan -av

Sandip

[Tom Weber_4]

I recreated the /etc/lvmtab by renaming the current lvmtab and running vgscan -av but the disk still appears and it shows the same status.

[John Poff]

Tom,

The vgcfgbackup/vgcfgrestore should backup and restore the LVM structures on that disk. It might be worth a try to see if that cleans it up.

JP

[Ashwani Kashyap]

DOes this physical volume whic you are trying to remove have multiple links , then remove all the links first and the ndo vgreduce .

Also check if there is an entry for this PV in /etc/lvmpvg . Try removing that entry and then do vgreduce .

Is there a way u can power off this LUN or all the disks in this LUN . If you can do that , then use the -f option with vgreduce to remove that PV completely from the VG . Once its done power them on .

And lastly have you checked for any latest LVM or scsi patches that your system might need .

[Tom Weber_4]

Ashwani, I tried powering off the LUN and running vgreduce -f /dev/vg01 and this was the result which may give a clue:

vgreduce: Invalid LVMREC on Physical Volume /dev/rdsk/c0t0d2

[Ashwani Kashyap]

Tom ,

So the vgreduce failed again . Usually an invaled LVMREC means that there is a bad header , maybe because of a bad disk . To ensure that a LUN is not bad do this ,
#dd if=/dev/rdsk/c0t0d2 of=/dev/null bs=1024 count=20

and see if number of record in and out are equal . IF not then you are dealing with a bad disk . In that case replace the disk , run a vgcfgrestore and then vgreduce .

Or just replace it , pvcreate it and then use vgreduce -f .

[Ashwani Kashyap]

Tom ,

Try running vgreduce -f , while its is powered on .

[Tom Weber_4]

I tried the dd which came out fine. I also did the vgredure -f with the array powered on and still no change. :(

The pvdisplay still shows 250 PE allocated as before.

[John Poff]

Hi again,

Since you want to get rid of the disk, how about trying the vgcfgrestore to it just to see if that cleans up the LVM info on it? Nothing to lose if it doesn't work, right?

JP

[Ashwani Kashyap]

TOm try this .

vgexport the entire VG into a map file .
# vgchange -a n /dev/vg01
# vgexport -m /dev/vg01 .
This should also remove /dev/vg01 directory .

Recreate /dev/vg01 directory .

#mknod /dev/vg01/group c 64 0xabcdef
Where abcdef is in hex and unique to your system .

Then vgimport VG01 on to all disks .
#vgimport -m /dev/vg01 /dev/dsk/c.t.d. /dev/dsk/c.t.d. ( All the PV's in that VG , make a note of it earlier . Use vgdisplay or strings on lvmtab )

vgimport should recreate the damaged headers .
#vgchnage -a y /dev/vg01 .

Then use vgreduce on that disk .

[Tom Weber_4]

John, I did try the vgcfgrestore and am left with the same results.

Ashwani, I'll have to schedule downtime to try the vgexport/vgimport method which I was trying to avoid.

Yes, all I need to do is delete this LUN and we will be removing this array entirely from our system. So it's not an issue that's critical but it would be great to get it cleaned up so we could remove the old array.

I'll award points after I see if anyone else comes up with ideas.

Thanks for the suggestions guys and anyone else who may come up with another option!

[John Poff]

Tom,

What happens if you try to create a new LV using all the PEs on that disk? If it complains about not having enough PEs, try making the LV with 250 PEs less than the max. If that works, maybe a pvdisplay -v will at least show you which 250 PEs it thinks are in use as they won't have your new LV assigned to them.

I just love playing with stuff that is broken, especially when it is on somebody elses system! ;)

JP

[Tom Weber_4]

Hey John, appreciate the efforts. Here's what I tried. I created an LV called TESTlv and used up all the PE on the disk shown below. Notice the free PE and allocated PE numbers were really out of whack after I did this:


--- Physical volumes ---
PV Name /dev/dsk/c0t0d2
VG Name /dev/vg01
PV Status available
Allocatable yes
VGDA 2
Cur LV 1
PE Size (Mbytes) 4
Total PE 7226
Free PE 65286
Allocated PE 4294909236
Stale PE 0
IO Timeout (Seconds) default

--- Distribution of physical volume ---
LV Name LE of LV PE for LV
/dev/vg01/TESTlv 7226 7226

--- Physical extents ---
PE Status LV LE
0000 current /dev/vg01/TESTlv 0000
0001 current /dev/vg01/TESTlv 0001
0002 current /dev/vg01/TESTlv 0002
........
7222 current /dev/vg01/TESTlv 7222
7223 current /dev/vg01/TESTlv 7223
7224 current /dev/vg01/TESTlv 7224
7225 current /dev/vg01/TESTlv 7225



I then removed the logical volume again which brought me back to square one:

--- Physical volumes ---
PV Name /dev/dsk/c0t0d2
VG Name /dev/vg01
PV Status available
Allocatable yes
VGDA 2
Cur LV 0
PE Size (Mbytes) 4
Total PE 7226
Free PE 6976
Allocated PE 250
Stale PE 0
IO Timeout (Seconds) default


--- Physical extents ---
PE Status LV LE
0000 free 0000
0001 free 0000
0002 free 0000
0003 free 0000
0004 free 0000
0005 free 0000

So the system still thinks there is 250 PE allocated and there isn't. This is getting to be quite an annoying problem. Any other ideas John or some else??? I just want to blow this LUN away anyway we can do it.

Thanks again, Tom.

[John Poff]

Tom,

When you powered off that disk and did the vgreduce -f it gave you the invalid LVMREC error. Here is what you could try, but I wouldn't recommend it unless you can take a chance on crashing something. I suggest doing a pvcreate -f on that disk, which should write new LVM structures to it. Then, power it off and try the vgreduce -f on it. Maybe the pvcreate will give it a clean LVMREC and make it happy enough for the vgreduce -f to work.

That is a pretty crazy solution, and like I said, I wouldn't try it unless you've got things backed up good and you can chance crashing things. Otherwise, I guess you have to get some downtime and try the vgexport/vgimport. Are you logging any errors for that disk in syslog or from EMS?

JP