Problem with vgextend and device busy error

[Karen Birkelbach]

I have an Integrity Virtual Machine running on an RX4640, VM software is very old. I have several clients with connections to two EVA's at different sites. The VM clients were originally set up on one EVA, then a mirror was added from the second EVA so that we could move the physical host server to the other site. I followed the procedure to mirror the boot disk, but was unable to test it properly before the move. Now, the server will only boot off the original EVA and while trying to test booting off the second, I have managed to lose information from the first EVA.

I removed the first-EVA paths from the client, found it wouldn't boot from the second EVA, added the first-EVA paths back into the client (using hpvmmodify) and rebooted, but all the device paths in the VM have now changed. I am unable to add them back into vg00 with the following error:

# vgextend vg00 /dev/dsk/c0t3d0s2
vgextend: Warning: Max_PE_per_PV for the volume group (4589) too small for this PV (4643).
Using only 4589 PEs from this physical volume.
vgextend: Couldn't install the physical volume "/dev/dsk/c0t3d0s2".
Device busy

Output from vgdisplay:

# vgdisplay -v vg00
--- Volume groups ---
VG Name /dev/vg00
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 13
Open LV 13
Max PV 16
Cur PV 4
Act PV 2
Max PE per PV 4589
VGDA 4
PE Size (Mbytes) 32
Total PE 9178
Alloc PE 6152
Free PE 3026
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg00/lvol1
LV Status available/stale
LV Size (Mbytes) 1024
Current LE 32
Allocated PE 64
Used PV 1

LV Name /dev/vg00/lvol2
LV Status available/stale
LV Size (Mbytes) 4096
Current LE 128
Allocated PE 256
Used PV 1

LV Name /dev/vg00/lvol3
LV Status available/stale
LV Size (Mbytes) 1024
Current LE 32
Allocated PE 64
Used PV 1

LV Name /dev/vg00/lvol4
LV Status available/stale
LV Size (Mbytes) 1024
Current LE 32
Allocated PE 64
Used PV 1

LV Name /dev/vg00/lvol5
LV Status available/stale
LV Size (Mbytes) 2048
Current LE 64
Allocated PE 128
Used PV 1

LV Name /dev/vg00/lvol6
LV Status available/stale
LV Size (Mbytes) 6016
Current LE 188
Allocated PE 376
Used PV 1

LV Name /dev/vg00/lvol7
LV Status available/stale
LV Size (Mbytes) 4320
Current LE 135
Allocated PE 270
Used PV 1

LV Name /dev/vg00/lvol8
LV Status available/stale
LV Size (Mbytes) 4608
Current LE 144
Allocated PE 288
Used PV 1

LV Name /dev/vg00/lvCDs
LV Status available/stale
LV Size (Mbytes) 12000
Current LE 375
Allocated PE 750
Used PV 1

LV Name /dev/vg00/lvsapmnt
LV Status available/stale
LV Size (Mbytes) 6016
Current LE 188
Allocated PE 376
Used PV 1

LV Name /dev/vg00/lvusrsap
LV Status available/stale
LV Size (Mbytes) 15008
Current LE 469
Allocated PE 938
Used PV 1

LV Name /dev/vg00/lvsapdb
LV Status available/stale
LV Size (Mbytes) 144000
Current LE 4500
Allocated PE 9000
Used PV 1

LV Name /dev/vg00/lvswap
LV Status available/stale
LV Size (Mbytes) 16000
Current LE 500
Allocated PE 1000
Used PV 1


--- Physical volumes ---
PV Name /dev/dsk/c0t1d0
PV Status available
Total PE 4589
Free PE 724
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c0t2d0s2
PV Status available
Total PE 4589
Free PE 2302
Autoswitch On
Proactive Polling On


Is there any way to recover my lost storage so that I can continue trying to get the server to reside on only one EVA? Any assistance would be greatly appreciated.

[Armin Kunaschik · Accepted solution]

When you use hpvmmodify without explicitly defining the instance numbers, the sequence is important because the instance numbers are assigned automatically.
To fix this the easiest way is to edit the config manually (/var/opt/hpvm/guests//vmm_config.current).
Stop the VM before doing so and issue hpvmmodify -F -P to force the changes.
Your problem might not only be related to that, but you did not post you vm configuration (hpvmstatus -P ) so additional data is necessary...

My 2 cents,
Armin

[Karen Birkelbach]

Here is the output from hpvmstatus -P

[Virtual Machine Details]
Virtual Machine Name VM # OS Type State
==================== ===== ======= ========
dxi-da-0 3 HPUX On

[Authorized Administrators]
Oper Groups:
Admin Groups:
Oper Users:
Admin Users:

[Virtual CPU Details]
Number Entitle
vCPUs Type Entitlement Maximum
====== ======= =========== =======
2 percent 50.0% 100.0%

[Memory Details]
Total Reserved
Memory Memory
=========== ===========
12 GB 64 MB

[Storage Interface Details]
Guest Physical
Device Adaptor Bus Dev Ftn Tgt Lun Storage Device
====== ========== === === === === === ========= =========================
disk scsi 0 1 0 0 0 disk /dev/rdsk/c16t0d1
disk scsi 0 1 0 1 0 lv /dev/vgdxi0/rlvdxi0edc1
disk scsi 0 1 0 2 0 lv /dev/vgdxi0/rlvdxi0edc2
disk scsi 0 1 0 3 0 disk /dev/rdsk/c15t0d1
disk scsi 0 1 0 4 0 disk /dev/rdsk/c17t0d1
disk scsi 0 1 0 5 0 disk /dev/rdsk/c18t0d1
disk scsi 0 1 0 6 0 disk /dev/rdsk/c18t0d2
disk scsi 0 1 0 7 0 disk /dev/rdsk/c17t0d2
disk scsi 0 1 0 8 0 disk /dev/rdsk/c16t0d2
disk scsi 0 1 0 9 0 disk /dev/rdsk/c15t0d2

[Network Interface Details]
Interface Adaptor Name/Num Bus Dev Ftn Mac Address
========= ========== ========== === === === =================
vswitch lan vmsw1 0 0 0 12-f9-26-c1-d7-3b
vswitch lan vmbackup 0 4 0 5a-15-dd-be-65-b0

[Misc Interface Details]
Guest Physical
Device Adaptor Bus Dev Ftn Tgt Lun Storage Device
====== ========== === === === === === ========= =========================
serial com1 tty console

[Armin Kunaschik]

You did not loose any data.. just the mirror.
Did you mirror the disk inside the VM? Looks like.

c0t3d0 is probably this disk:
disk scsi 0 1 0 3 0 disk /dev/rdsk/c15t0d1
and it's smaller than your bootdisk (MaxPE!)

This is your boot disk:
disk scsi 0 1 0 2 0 lv /dev/vgdxi0/rlvdxi0edc2

But it's on LVM... and this is the right place to mirror disks in a HPVM setup.
From the setup I'd suspect c0t1d0 as mirror target then, but this does not make sense because it's in the same volume group but a different logical volume.
c0t3d0 is a "normal" raw device and probably not the disk you want to use for the root mirror. Your instance numbers look kind of mixed. Before doing anything, make sure, you're dealing with the right disks!
Do you have the tool evainfo installed? It's a very helpful tool to find out everything about EVA luns.

My advice: Move the mirror disk out of the VM and do the mirror on the host, e.g. put the disk into vgdxi0 and mirror rlvdxi0ed2.

If you want to restore the old setup, find out what LUN was presented with what instance number into the VM. The backup vmm_config.last might be helpful here.
Then restore the old setup by editing vmm_config.current manually.

And finally, when the VM version is quite old, consider an upgrade, at least for performance reason (AVIO etc).

My 2 cents,
Armin

PS: Please assign points if you find answers useful!

[Karen Birkelbach]

Thanks for ALL the help. Based on the first response about editing the vmm_config file, I decided to restore it because I was having trouble translating the device information that had saved previously (yes, I'll work on that). I restored the file and was able to boot the VM back to the way it was before I started down this path.

This is the original VM Server here and it's on an old version of HP-UX 11.23. I've not been successful with upgrading it, but in a couple of weeks, I'm taking all of the VM servers to at least 3.5, 4.0 on the newest one.

Amin, I will review your suggestions and use them to help me get the boot disk mirrored and get the VM off the original EVA.

Thank you!