LVM and VxVM

Disk already in use by another logical volume

 
Les Harrison
Advisor

Disk already in use by another logical volume

Hi

Question. New disk was presented from our SAN (EMC).
I ran insf –e and we were able to see that disk.

Running ioscan I was able to see the physical volumes assigned to that disk. On server 1 they were:



> /dev/dsk/c14t2d0 /dev/rdsk/c14t2d0

> /dev/dsk/c16t2d0 /dev/rdsk/c16t2d0



I want to add these disks to vgclord volume group.

When I tried to pvcreate these it said they already belonged to a volume group.

I checked /etc/lvmtab but they were not there. These two servers are in a Service Guard cluster (trying to add disks to different packages under that cluster) Could this be a cluster issue?

I checked /etc/lvmtab on Server 2 in the cluster and these volumes are under the vgclwss volume group.

Can I do anything like re-assign different physical volumes to this disk from the SAN? Or is there an alternative?

Thanks

Les
13 REPLIES 13

Re: Disk already in use by another logical volume

Just stop and be careful here...

How have you estableshed that they are the same disks between the two cluster nodes... simply having the same device file name does not mean they are the same disk.

Better to post the output of:

ioscan -funCdisk

and let us figure out which disks are which...

HTH

Duncan

I am an HPE Employee
Accept or Kudo
Les Harrison
Advisor

Re: Disk already in use by another logical volume

Hi Duncan

Long time no hear. See attached for server 1 ioscan.
Les Harrison
Advisor

Re: Disk already in use by another logical volume

And server 2 ioscan attached...

Re: Disk already in use by another logical volume

Les,

OK, looking at these outputs, you need to look at the part of the hardware path that comes after the FC HBA.

From this I can determine that these boxes each have 2 FC ports connected to 2 ports on your DMX.

SO I can see that:

- disk devices which are presented from a port on the DMX which has hardware path 108.102.19 all start with c14 on server 1 and c12 on server 2.

- disk devices which are presented from a port on the DMX which has hardware path 107.110.19 all start with c16 on server 1 and c14 on server 2.

I'd imagine that the two paths are to the same sets of LUNs.

The disks you mention certainly have the highest instance number (77 and 78 on server 1 and 70 and 71 on server 2), so they certainly look like the last 2 that were added - maybe they have already been presented somehwere else before. You can check they're not already in use on these machines by running:

on server 1:
pvdisplay -v /dev/dsk/c14t2d0
pvdisplay -v /dev/dsk/c16t2d0
strings /etc/lvmtab

on server 2:
pvdisplay -v /dev/dsk/c12t2d0
pvdisplay -v /dev/dsk/c14t2d0
strings /etc/lvmtab

Even if they're not in use on these systems, if they already appear to have an LVM structure on them (as the pvcreate serror suggests), I'd at least have your SAN team check again that they haven't presented you with a LUN from another HPUX box.

HTH

Duncan

I am an HPE Employee
Accept or Kudo
Les Harrison
Advisor

Re: Disk already in use by another logical volume

Hello Duncan

Sorry for delay.

Server 1:

root@ugos389# pvdisplay -v /dev/dsk/c14t2d0
pvdisplay: Couldn't find the volume group to which
physical volume "/dev/dsk/c14t2d0" belongs.
pvdisplay: Cannot display physical volume "/dev/dsk/c14t2d0".
root@ugos389# pvdisplay -v /dev/dsk/c16t2d0
pvdisplay: Couldn't find the volume group to which
physical volume "/dev/dsk/c16t2d0" belongs.
pvdisplay: Cannot display physical volume "/dev/dsk/c16t2d0".
root@ugos389# strings /etc/lvmtab
/dev/vg00
Bi\G
/dev/dsk/c2t1d0s2
/dev/dsk/c3t0d0s2
/dev/vgapp
/dev/dsk/c14t0d2
/dev/dsk/c14t1d4
/dev/dsk/c16t0d2
/dev/dsk/c16t1d4
/dev/vgclord
/dev/dsk/c14t0d7
/dev/dsk/c14t5d7
/dev/dsk/c16t0d7
/dev/dsk/c16t5d7
/dev/dsk/c16t1d6
/dev/dsk/c14t1d6
/dev/vgclwss
/dev/dsk/c14t0d3
/dev/dsk/c14t0d4
/dev/dsk/c14t0d5
/dev/dsk/c14t0d6
/dev/dsk/c14t1d0
/dev/dsk/c14t1d1
/dev/dsk/c14t1d2
/dev/dsk/c14t1d3
/dev/dsk/c14t1d7
/dev/dsk/c14t3d5
/dev/dsk/c16t0d3
/dev/dsk/c16t0d4
/dev/dsk/c16t0d5
/dev/dsk/c16t0d6
/dev/dsk/c16t1d0
/dev/dsk/c16t1d1
/dev/dsk/c16t1d2
/dev/dsk/c16t1d3
/dev/dsk/c16t1d7
/dev/dsk/c16t3d5
Les Harrison
Advisor

Re: Disk already in use by another logical volume

Server 2 (also see attached):

root@ugos388# strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2t1d0s2
/dev/dsk/c3t0d0s2
/dev/vgapp
/dev/dsk/c12t0d1
/dev/dsk/c12t1d5
/dev/dsk/c14t0d1
/dev/dsk/c14t1d5
/dev/vgclwss
/dev/dsk/c12t0d3
/dev/dsk/c12t0d4
/dev/dsk/c12t0d5
/dev/dsk/c12t0d6
/dev/dsk/c12t1d0
/dev/dsk/c12t1d1
/dev/dsk/c12t1d2
/dev/dsk/c12t1d3
/dev/dsk/c12t3d5
/dev/dsk/c14t0d3
/dev/dsk/c14t0d4
/dev/dsk/c14t0d5
/dev/dsk/c14t0d6
/dev/dsk/c14t1d0
/dev/dsk/c14t1d1
/dev/dsk/c14t1d2
/dev/dsk/c14t1d3
/dev/dsk/c14t3d5
/dev/dsk/c12t1d7
/dev/dsk/c14t1d7
/dev/dsk/c12t2d0
/dev/dsk/c14t2d0
/dev/vgclord
/dev/dsk/c12t0d7
/dev/dsk/c12t1d6
/dev/dsk/c12t5d7
/dev/dsk/c14t0d7
/dev/dsk/c14t1d6
/dev/dsk/c14t5d7
Matti_Kurkela
Honored Contributor

Re: Disk already in use by another logical volume

If the disk presented from the SAN was previously used by another HP-UX and was not thoroughly wiped, it might still contain the LVM header created by that previous system.

The pvcreate message just means "Someone has issued pvcreate on this disk already." If you wish to avoid this, you should start using the pvremove command before unpresenting old disks.

Check your SAN configuration: if the disks are in use by some other host that does not belong in your cluster, overwriting the disk will corrupt the data used by that other host.

You should install the "syminq" (or sometimes just "inq") tool from EMC: it can find the EMC-specific disk ID numbers from the SAN disks. The ioscan output is of limited utility here: without seeing the EMC identifiers you cannot be 100% sure the disk presented on system A is the same disk as presented on system B. As you've discovered, the hardware paths can (and often will) be different between systems.

If you have the symcli tools installed, "sympd list" can be used instead of "syminq".

You have already confirmed your cluster is not using these disks. If you can make sure the disks are presented to your cluster *only*, you can safely use "pvcreate -f" to get rid of the old LVM information that seems to be on these disks.

MK
MK
Ivan Krastev
Honored Contributor

Re: Disk already in use by another logical volume

Check LVM headers for both disks (c1t2d0 is used for example):
#xd -j8200 -N16 -tu /dev/rdsk/c1t2d0
0000000 2000252410 965817345 2000252410 965817462
PV CPU-ID PV timestamp VG CPU-ID VG timestamp
The above information translates to:
? pvcreate and vgcreate was run on the sytem with systemID (uname -i) 2000252410
? pvcreate was run at timestamp 965817345 (seconds after Jan 1st 1970 0:00 UTC)
? vgcreate was run at timestamp 965817462 (117 seconds later)


You fill find system ID (uname -i) of the system and VG ID as well.


regards,
ivan
Les Harrison
Advisor

Re: Disk already in use by another logical volume

Hi guys,

I ran the xd commands of server 1:

root@ugos389# xd -j8200 -N16 -tu /dev/rdsk/c14t2d0
0000000 2726266024 1208699356 2726266024 1126011188
0000010
root@ugos389# xd -j8200 -N16 -tu /dev/rdsk/c16t2d0
0000000 2726266024 1208699356 2726266024 1126011188
0000010

root@ugos389# uname -i
3752607443

On server 2:

$ uname -i
2726266024

So it looks like they were never pvcreated on server 1, just on server 2.

I'm thinking that these disks must be registered in service guard.
Looking at the Service Guard cluster config I can't see the physical volumes referenced on either server but the volume group vgclwss is:
root@ugos388# grep FIRST_CLUSTER_LOCK_VG *
cluster.ascii:# Lock Disk Parameters. Use the FIRST_CLUSTER_LOCK_VG and
cluster.ascii:# The FIRST_CLUSTER_LOCK_VG is the LVM volume group that
cluster.ascii:FIRST_CLUSTER_LOCK_VG /dev/vgclwss
wssprodcl.conf:# Lock Disk Parameters. Use the FIRST_CLUSTER_LOCK_VG and
wssprodcl.conf:# The FIRST_CLUSTER_LOCK_VG is the LVM volume group that
wssprodcl.conf:FIRST_CLUSTER_LOCK_VG /dev/vgclwss
wssprodcl.conf.011206:# Lock Disk Parameters. Use the FIRST_CLUSTER_LOCK_VG and
wssprodcl.conf.011206:# The FIRST_CLUSTER_LOCK_VG is the LVM volume group that
wssprodcl.conf.011206:FIRST_CLUSTER_LOCK_VG /dev/vgclwss
root@ugos388# grep FIRST_CLUSTER_LOCK_PV *
cluster.ascii:# FIRST_CLUSTER_LOCK_PV parameters to define a lock disk.
cluster.ascii: FIRST_CLUSTER_LOCK_PV /dev/dsk/c8t0d3
cluster.ascii: FIRST_CLUSTER_LOCK_PV /dev/dsk/c10t0d3
wssprodcl.conf:# FIRST_CLUSTER_LOCK_PV parameters to define a lock disk.
wssprodcl.conf:FIRST_CLUSTER_LOCK_PV /dev/dsk/c12t0d3
wssprodcl.conf:FIRST_CLUSTER_LOCK_PV /dev/dsk/c14t0d3
wssprodcl.conf.011206:# FIRST_CLUSTER_LOCK_PV parameters to define a lock disk.
wssprodcl.conf.011206:FIRST_CLUSTER_LOCK_PV /dev/dsk/c8t0d3
wssprodcl.conf.011206:FIRST_CLUSTER_LOCK_PV /dev/dsk/c10t0d3
root@ugos388# grep c14t2d0 *
root@ugos388#