Problem in LVM

[Suresh Babu G]

Hi Experts,

We have connected one MSA30 (Toal of 14 Disks, 136GB of capacity each) in one of the server and when we try to add a disk into the VG it is giving error, already this PV was participating in one of the VG and we have removed that VG.

# pvcreate /dev/rdsk/c15t15d0
pvcreate: The physical volume already belongs to a volume group

# pvcreate -f /dev/rdsk/c15t15d0
pvcreate: writing LVM record: Permission denied

# pvremove /dev/rdsk/c15t15d0
pvremove: The physical volume "/dev/rdsk/c15t15d0" belongs to an exported volume group.
pvremove: Couldn't remove physical volume "/dev/rdsk/c15t15d0".

# vgextend vgbkp1 /dev/dsk/c15t15d0
vgextend: Couldn't install the physical volume "/dev/dsk/c15t15d0".
Device busy
#

How it can be resolved, Please suggest.

Warm Regards
Suresh

[Torsten.]

Is this a cluster environment?

[Suresh Babu G]

Hi,

It is not in cluster environment

Warm Regards
Suresh.

[Shibin_2]

What's the output of pvdisplay /dev/dsk/c15t15d0 ?

[Suresh Babu G]

Hi,

here is the pvdisplay output

# pvdisplay /dev/dsk/c15t15d0
pvdisplay: Couldn't find the volume group to which
physical volume "/dev/dsk/c15t15d0" belongs.
pvdisplay: Cannot display physical volume "/dev/dsk/c15t15d0".
#

Warm Regards
Suresh.

[Joachim Noffke]

Looks like something went wrong when you removed the old volume group. Have you examined the output from "strings /etc/lvmtab" to see if the old VG and/or the disk are still present?

[Manix]

Hello Suresh !!

you may run vgscan -a -v -p ( preview mode )
and check if there are some inconsistencies
with the VGs you want.

These disk have the VGID /PVID associated with the previous volume group.

Depends how you removed that VG , just exported it or destroyed it completely step by step.

The way you did may help in solving it further.

Thanks & Regards

Manix

[Suresh Babu G]

hi,

the strings /etc/lvmtab doesn't give that particular PV information

# strings /etc/lvmtab
/dev/eap3
dL3)
/dev/dsk/c3t14d0
/dev/vg01
/dev/dsk/c3t13d0
/dev/vg00
/dev/disk/disk37_p2
/dev/dsk/c2t6d0s2
/dev/vgbkp1
/dev/dsk/c15t0d0
/dev/dsk/c15t9d0
/dev/dsk/c15t1d0
/dev/dsk/c15t2d0
/dev/dsk/c15t3d0
/dev/dsk/c15t4d0
/dev/dsk/c15t5d0
/dev/dsk/c15t8d0
/dev/dsk/c15t10d0
/dev/dsk/c15t11d0
/dev/dsk/c15t12d0
/dev/dsk/c15t13d0
/dev/dsk/c15t14d0
#

Warm Regards
Suresh.

[Suresh Babu G]

hi,

here is the vgscan with preview mode as attachment

Following Physical Volumes belong to one Volume Group.
Unable to match these Physical Volumes to a Volume Group.
Use the vgimport command to complete the process.
/dev/dsk/c15t15d0

Warm Regards
Suresh.

PS: we have removed the VG in a proper manner as below.

lvremove
vgreduce
vgremove

[Manix]

so this PV is not claimed in vgscan /lvmtab
.This may confirm that its not a part of existing VG ..& i guess this is not a alternate link as shown in "vgdisplay -v |grep -i 'PV Name'

Then we may run dd command to remove the disk headers and create PVs again.

Give it a go !!

[Manix]

Hey !!

Please ignore the previous post as i haven`t seen the file which you just attached.

Thanks

Manix

[Manix]

Looking at the output we may see that there are no of LUNS which belongs or belonged to other VGS , possibly this MSA was used else where.

The best way would be to recover these VGS and destroy them ,step wise

# mkdir /dev/vg11
# mknod /dev/vg11/group c 64 0x010011
#vgimport -v /dev/vg01 /dev/dsk/c0t1d0 /dev/dsk/c0t3d0

then destroy as you did earlier.
lvdestroy, vgreduce , vgdestroy, pvremove.

Looks this MSA has good lot`s of things.

Thanks

Manix

[Sreer]

Hi,

First of all "PLS ASSIGN POINTS" to ones who are helping and spending some time for you !!!

While coming to the issue.

As maxin explained the Disks can be used for some other Vgs.
So if you are sure that these PVs are no more in use in any system.

Pls do Vgscan again.

still errors are existing find out whether the disks are having parallel conection to any other boxes?

is it possible to have a reboot? if vgscan fails to solve the issue?

rgds
Sree

[Zinky]

IF you are absolutely sure the disks in your MSA30 are fully free (regardless of however they were previously used), then do the following for EACH disk:

dd id=/dev/zero of=/dev/rdsk/cXtYdZ bs=1024k count=100

That WILL effectively wipeout remnants of any configuration - LVM/VxVM/filesystem,etc -- so you can smoothly use them under LVM (pvcreate) or VxVM (vxdisksetup)

Hope this helps you Babu.

[Jeff Gyurko]

A couple of things concern me here that should be clarified BEFORE you do anything. You have a lot of output from vgscan that shows devices that are not correctly removed from service. If you do as you say, lvreduce, lvremove, vgreduce vgremove...you should not get the warnings from vgscan that states so many disks belong to a volume group. The LVM information is removed when you follow the correct proces.

As one suggested, I would follow the process of importing a VG (mkdir, mknod, vgimport, vgchange, mount) on the disk in question (c15t15d0) and find out what it is. When the device busy message comes up and you're certain you are not using it on this server, it makes me think they are utilized on another server. Find out what it is. If you still get the "device busy" errors I would verify with any other server that is connected to that array.

Check the output of /usr/contrib/bin/inq if it exists as sometimes they give device serial numbers. Check this number from the other servers that might be attached to the array.

Try getting some data off the device (dd if=/dev/dsk/c15t15d0 of=/tmp/output count=1000) See what's in the output.

Above all, be certain that anything destructive, (pvcreate -f, dd is done after all possible checking is done).

[Suresh Babu G]

Hi Experts,

Thanks to everyone and the problem got resolved after replacing the disk.

Warm Regards
Suresh

[Zinky]

If yer sure that MSA30 all have reusable drives,

If yer sure your cXtYdZ's you intending to use to carve a new VG point to disks in that SCSI enclosure/array,

Just simply follow the "dd" trickology above and get on your way.

[Suresh Babu G]

It started working after replacing the disk