topic Re: Problem in LVM in Operating System - HP-UX

Problem in LVM

Suresh Babu G — Tue, 11 Jan 2011 04:57:38 GMT

Hi Experts,

We have connected one MSA30 (Toal of 14 Disks, 136GB of capacity each) in one of the server and when we try to add a disk into the VG it is giving error, already this PV was participating in one of the VG and we have removed that VG.

# pvcreate /dev/rdsk/c15t15d0
pvcreate: The physical volume already belongs to a volume group

# pvcreate -f /dev/rdsk/c15t15d0
pvcreate: writing LVM record: Permission denied

# pvremove /dev/rdsk/c15t15d0
pvremove: The physical volume "/dev/rdsk/c15t15d0" belongs to an exported volume group.
pvremove: Couldn't remove physical volume "/dev/rdsk/c15t15d0".

# vgextend vgbkp1 /dev/dsk/c15t15d0
vgextend: Couldn't install the physical volume "/dev/dsk/c15t15d0".
Device busy
#

How it can be resolved, Please suggest.

Warm Regards
Suresh

Re: Problem in LVM

Torsten. — Tue, 11 Jan 2011 06:39:47 GMT

Is this a cluster environment?

Re: Problem in LVM

Suresh Babu G — Tue, 11 Jan 2011 06:44:02 GMT

Hi,

It is not in cluster environment

Warm Regards
Suresh.

Re: Problem in LVM

Shibin_2 — Tue, 11 Jan 2011 09:01:08 GMT

What's the output of pvdisplay /dev/dsk/c15t15d0 ?

Re: Problem in LVM

Suresh Babu G — Tue, 11 Jan 2011 09:06:33 GMT

Hi,

here is the pvdisplay output

# pvdisplay /dev/dsk/c15t15d0
pvdisplay: Couldn't find the volume group to which
physical volume "/dev/dsk/c15t15d0" belongs.
pvdisplay: Cannot display physical volume "/dev/dsk/c15t15d0".
#

Warm Regards
Suresh.

Re: Problem in LVM

Joachim Noffke — Tue, 11 Jan 2011 11:22:15 GMT

Looks like something went wrong when you removed the old volume group. Have you examined the output from "strings /etc/lvmtab" to see if the old VG and/or the disk are still present?

Re: Problem in LVM

Manix — Tue, 11 Jan 2011 11:32:08 GMT

Hello Suresh !!

you may run vgscan -a -v -p ( preview mode )
and check if there are some inconsistencies
with the VGs you want.

These disk have the VGID /PVID associated with the previous volume group.

Depends how you removed that VG , just exported it or destroyed it completely step by step.

The way you did may help in solving it further.

Thanks & Regards

Manix

Re: Problem in LVM

Suresh Babu G — Tue, 11 Jan 2011 11:59:42 GMT

hi,

the strings /etc/lvmtab doesn't give that particular PV information

# strings /etc/lvmtab
/dev/eap3
dL3)
/dev/dsk/c3t14d0
/dev/vg01
/dev/dsk/c3t13d0
/dev/vg00
/dev/disk/disk37_p2
/dev/dsk/c2t6d0s2
/dev/vgbkp1
/dev/dsk/c15t0d0
/dev/dsk/c15t9d0
/dev/dsk/c15t1d0
/dev/dsk/c15t2d0
/dev/dsk/c15t3d0
/dev/dsk/c15t4d0
/dev/dsk/c15t5d0
/dev/dsk/c15t8d0
/dev/dsk/c15t10d0
/dev/dsk/c15t11d0
/dev/dsk/c15t12d0
/dev/dsk/c15t13d0
/dev/dsk/c15t14d0
#

Warm Regards
Suresh.

Re: Problem in LVM

Suresh Babu G — Tue, 11 Jan 2011 12:03:28 GMT

hi,

here is the vgscan with preview mode as attachment

Following Physical Volumes belong to one Volume Group.
Unable to match these Physical Volumes to a Volume Group.
Use the vgimport command to complete the process.
/dev/dsk/c15t15d0

Warm Regards
Suresh.

PS: we have removed the VG in a proper manner as below.

lvremove
vgreduce
vgremove

Re: Problem in LVM

Manix — Tue, 11 Jan 2011 12:12:31 GMT

so this PV is not claimed in vgscan /lvmtab
.This may confirm that its not a part of existing VG ..& i guess this is not a alternate link as shown in "vgdisplay -v |grep -i 'PV Name'

Then we may run dd command to remove the disk headers and create PVs again.

Give it a go !!

Re: Problem in LVM

Manix — Tue, 11 Jan 2011 12:15:48 GMT

Hey !!

Please ignore the previous post as i haven`t seen the file which you just attached.

Thanks

Manix

Re: Problem in LVM

Manix — Tue, 11 Jan 2011 12:31:32 GMT

Looking at the output we may see that there are no of LUNS which belongs or belonged to other VGS , possibly this MSA was used else where.

The best way would be to recover these VGS and destroy them ,step wise

# mkdir /dev/vg11
# mknod /dev/vg11/group c 64 0x010011
#vgimport -v /dev/vg01 /dev/dsk/c0t1d0 /dev/dsk/c0t3d0

then destroy as you did earlier.
lvdestroy, vgreduce , vgdestroy, pvremove.

Looks this MSA has good lot`s of things.

Thanks

Manix

Re: Problem in LVM

Sreer — Tue, 11 Jan 2011 12:46:35 GMT

Hi,

First of all "PLS ASSIGN POINTS" to ones who are helping and spending some time for you !!!

While coming to the issue.

As maxin explained the Disks can be used for some other Vgs.
So if you are sure that these PVs are no more in use in any system.

Pls do Vgscan again.

still errors are existing find out whether the disks are having parallel conection to any other boxes?

is it possible to have a reboot? if vgscan fails to solve the issue?

rgds
Sree

Re: Problem in LVM

Zinky — Tue, 11 Jan 2011 14:21:13 GMT

IF you are absolutely sure the disks in your MSA30 are fully free (regardless of however they were previously used), then do the following for EACH disk:

dd id=/dev/zero of=/dev/rdsk/cXtYdZ bs=1024k count=100

That WILL effectively wipeout remnants of any configuration - LVM/VxVM/filesystem,etc -- so you can smoothly use them under LVM (pvcreate) or VxVM (vxdisksetup)

Hope this helps you Babu.

Re: Problem in LVM

Jeff Gyurko — Thu, 13 Jan 2011 15:52:44 GMT

A couple of things concern me here that should be clarified BEFORE you do anything. You have a lot of output from vgscan that shows devices that are not correctly removed from service. If you do as you say, lvreduce, lvremove, vgreduce vgremove...you should not get the warnings from vgscan that states so many disks belong to a volume group. The LVM information is removed when you follow the correct proces.

As one suggested, I would follow the process of importing a VG (mkdir, mknod, vgimport, vgchange, mount) on the disk in question (c15t15d0) and find out what it is. When the device busy message comes up and you're certain you are not using it on this server, it makes me think they are utilized on another server. Find out what it is. If you still get the "device busy" errors I would verify with any other server that is connected to that array.

Check the output of /usr/contrib/bin/inq if it exists as sometimes they give device serial numbers. Check this number from the other servers that might be attached to the array.

Try getting some data off the device (dd if=/dev/dsk/c15t15d0 of=/tmp/output count=1000) See what's in the output.

Above all, be certain that anything destructive, (pvcreate -f, dd is done after all possible checking is done).

Re: Problem in LVM

Suresh Babu G — Thu, 13 Jan 2011 15:57:03 GMT

Hi Experts,

Thanks to everyone and the problem got resolved after replacing the disk.

Warm Regards
Suresh

Re: Problem in LVM

Zinky — Thu, 13 Jan 2011 15:57:48 GMT

If yer sure that MSA30 all have reusable drives,

If yer sure your cXtYdZ's you intending to use to carve a new VG point to disks in that SCSI enclosure/array,

Just simply follow the "dd" trickology above and get on your way.

Re: Problem in LVM

Suresh Babu G — Thu, 13 Jan 2011 16:00:17 GMT

It started working after replacing the disk