vgreduce won't work, physical extents in use supposedly

[Tom Weber_4]

Ashwani, I tried powering off the LUN and running vgreduce -f /dev/vg01 and this was the result which may give a clue:

vgreduce: Invalid LVMREC on Physical Volume /dev/rdsk/c0t0d2

[Ashwani Kashyap]

Tom ,

So the vgreduce failed again . Usually an invaled LVMREC means that there is a bad header , maybe because of a bad disk . To ensure that a LUN is not bad do this ,
#dd if=/dev/rdsk/c0t0d2 of=/dev/null bs=1024 count=20

and see if number of record in and out are equal . IF not then you are dealing with a bad disk . In that case replace the disk , run a vgcfgrestore and then vgreduce .

Or just replace it , pvcreate it and then use vgreduce -f .

[Ashwani Kashyap]

Tom ,

Try running vgreduce -f , while its is powered on .

[Tom Weber_4]

I tried the dd which came out fine. I also did the vgredure -f with the array powered on and still no change. :(

The pvdisplay still shows 250 PE allocated as before.

[John Poff]

Hi again,

Since you want to get rid of the disk, how about trying the vgcfgrestore to it just to see if that cleans up the LVM info on it? Nothing to lose if it doesn't work, right?

JP

[Ashwani Kashyap]

TOm try this .

vgexport the entire VG into a map file .
# vgchange -a n /dev/vg01
# vgexport -m /dev/vg01 .
This should also remove /dev/vg01 directory .

Recreate /dev/vg01 directory .

#mknod /dev/vg01/group c 64 0xabcdef
Where abcdef is in hex and unique to your system .

Then vgimport VG01 on to all disks .
#vgimport -m /dev/vg01 /dev/dsk/c.t.d. /dev/dsk/c.t.d. ( All the PV's in that VG , make a note of it earlier . Use vgdisplay or strings on lvmtab )

vgimport should recreate the damaged headers .
#vgchnage -a y /dev/vg01 .

Then use vgreduce on that disk .

[Tom Weber_4]

John, I did try the vgcfgrestore and am left with the same results.

Ashwani, I'll have to schedule downtime to try the vgexport/vgimport method which I was trying to avoid.

Yes, all I need to do is delete this LUN and we will be removing this array entirely from our system. So it's not an issue that's critical but it would be great to get it cleaned up so we could remove the old array.

I'll award points after I see if anyone else comes up with ideas.

Thanks for the suggestions guys and anyone else who may come up with another option!

[John Poff]

Tom,

What happens if you try to create a new LV using all the PEs on that disk? If it complains about not having enough PEs, try making the LV with 250 PEs less than the max. If that works, maybe a pvdisplay -v will at least show you which 250 PEs it thinks are in use as they won't have your new LV assigned to them.

I just love playing with stuff that is broken, especially when it is on somebody elses system! ;)

JP

[Tom Weber_4]

Hey John, appreciate the efforts. Here's what I tried. I created an LV called TESTlv and used up all the PE on the disk shown below. Notice the free PE and allocated PE numbers were really out of whack after I did this:


--- Physical volumes ---
PV Name /dev/dsk/c0t0d2
VG Name /dev/vg01
PV Status available
Allocatable yes
VGDA 2
Cur LV 1
PE Size (Mbytes) 4
Total PE 7226
Free PE 65286
Allocated PE 4294909236
Stale PE 0
IO Timeout (Seconds) default

--- Distribution of physical volume ---
LV Name LE of LV PE for LV
/dev/vg01/TESTlv 7226 7226

--- Physical extents ---
PE Status LV LE
0000 current /dev/vg01/TESTlv 0000
0001 current /dev/vg01/TESTlv 0001
0002 current /dev/vg01/TESTlv 0002
........
7222 current /dev/vg01/TESTlv 7222
7223 current /dev/vg01/TESTlv 7223
7224 current /dev/vg01/TESTlv 7224
7225 current /dev/vg01/TESTlv 7225



I then removed the logical volume again which brought me back to square one:

--- Physical volumes ---
PV Name /dev/dsk/c0t0d2
VG Name /dev/vg01
PV Status available
Allocatable yes
VGDA 2
Cur LV 0
PE Size (Mbytes) 4
Total PE 7226
Free PE 6976
Allocated PE 250
Stale PE 0
IO Timeout (Seconds) default


--- Physical extents ---
PE Status LV LE
0000 free 0000
0001 free 0000
0002 free 0000
0003 free 0000
0004 free 0000
0005 free 0000

So the system still thinks there is 250 PE allocated and there isn't. This is getting to be quite an annoying problem. Any other ideas John or some else??? I just want to blow this LUN away anyway we can do it.

Thanks again, Tom.

[John Poff]

Tom,

When you powered off that disk and did the vgreduce -f it gave you the invalid LVMREC error. Here is what you could try, but I wouldn't recommend it unless you can take a chance on crashing something. I suggest doing a pvcreate -f on that disk, which should write new LVM structures to it. Then, power it off and try the vgreduce -f on it. Maybe the pvcreate will give it a clean LVMREC and make it happy enough for the vgreduce -f to work.

That is a pretty crazy solution, and like I said, I wouldn't try it unless you've got things backed up good and you can chance crashing things. Otherwise, I guess you have to get some downtime and try the vgexport/vgimport. Are you logging any errors for that disk in syslog or from EMS?

JP