topic Re: Root disk LVM problem in Operating System - HP-UX

Root disk LVM problem

Tom Ward_1 — Mon, 23 May 2011 16:04:51 GMT

I have a rp3440 that had three disks in vg00.
c2t0d0, c2t2d0 and a SAN LUN c16t13d7. The OS was mirrored between the two local disks. This server doesn't follow typical vg00 layout.

The local disk has
lvol1 is stand
lvol2 is swap
lvol3 is /
lvol4 is /home

lvol5 is /u01 and is on the SAN LUN, but someone took a really bad shortcut and expanded lvol5 onto the main OS disk, c2t0d0.

c2t0d0 failed. /u01 never went unavailable, but we started getting a lot of SCSI errors and any commands that hit the part on c2t0d0 would hang.

I was able to reduce c2t0d0 from everything but lvol5.

I made a new VG and LV, restored /u01 to that and updated /etc/fstab to use the new /u01.

I tried to shutdown to replace the problem disk, but shutdown hung at unmount file systems. Eventually I did a TOC and swapped the disk. The system booted up fine and is using the new /u01.

Now my problem. vg00 doesn't show the disk with a vgdisplay, but the disk shows up in /etc/lvmtab. My attempts to reduce the disk out fail.

I've tried saving a copy of the lvmtab and rebuilding it with vgscan -v, but c2t0d0 doesn't come out.

Re: Root disk LVM problem

Fraga — Mon, 23 May 2011 16:23:20 GMT

Hi Tom,
could you explain how you created the new vg?

and:
ioscan -fnCdisk
strings /etc/lvmtab
vgdisplay vgXX

lvdisplay -v someLvol
pvdisplay disk.

regards,
EFS

Re: Root disk LVM problem

Fraga — Mon, 23 May 2011 16:25:55 GMT

I saw just now your attachment.

Re: Root disk LVM problem

Fraga — Mon, 23 May 2011 16:30:23 GMT

The problem is your mirror.
#lvdisplay -v -k /dev/vg00/lvolY

You need reduce it, for example:

#lvdisplay -v -k /dev/vg00/lvolY
...
--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 0 01094 stale 1 01094 current

#lvreduce -m 0 -k /dev/vg00/lvolY 0 ;where 0 = PV with status stale

Re: Root disk LVM problem

Tom Ward_1 — Mon, 23 May 2011 16:37:01 GMT

Hello EFS,

There was another unused 72GB local disk c3t2d0. I used that to make vg01

# mkdir /dev/vg01
# mknod /dev/vg01/group c 64 0x010000
# vgcreate -s 16 vg01 /dev/dsk/c3t2d0
# vgdisplay -v /dev/vg01
# lvcreate -l 1736 /dev/vg01
# lvdisplay /dev/vg01/lvol1
# newfs -F vxfs -o largefile /dev/vg01/rlvol1

Mount /dev/vg01/lvol1 as /u01_new
# mkdir /u01_new
# mount /dev/vg01/lvol1 /u01_new

Also following the reboot I removed the original vg00/lvol5.

# lvremove -f /dev/vg00/lvol5
lvremove: Couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
Current path "/dev/dsk/c18t13d7" is an alternate link, skip.
Logical volume "/dev/vg00/lvol5" has been successfully removed.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
vgcfgbackup: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Warning: couldn't query all of the physical volumes.
vgcfgbackup: Couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group

Re: Root disk LVM problem

Tom Ward_1 — Mon, 23 May 2011 16:50:33 GMT

There are no stale extents in the remaining vg00 lvols since I removed lvol5 after the system rebooted.

I just removed the SAN LUN, which shouldn't have been in vg00 to begin with. All the extents on that PV were showing free.

The warnings may give a clue as to what is wrong.

00023 free 00000
00024 free 00000
# vgreduce vg00 /dev/dsk/c16t13d7 /dev/dsk/c18t13d7
vgreduce: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
vgreduce: Warning: couldn't query all of the physical volumes.
Device file path "/dev/dsk/c16t13d7" is an primary link.
Removing primary link and switching to an alternate link.
Device file path "/dev/dsk/c18t13d7" is an alternate path.
Volume group "vg00" has been successfully reduced.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
vgcfgbackup: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Warning: couldn't query all of the physical volumes.
vgcfgbackup: Couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: /etc/lvmtab is out of date with the running kernel:Kernel indicates 1 disks for "/dev/vg00"; /etc/lvmta b has 2 disks.
Cannot proceed with backup.

Re: Root disk LVM problem

Tom Ward_1 — Mon, 23 May 2011 18:42:21 GMT

After I removed the SAN LUN from vg00 the lvmtab updated to show only the one local disk in vg00. I was then able to re-add the other local disk to vg00 using the procedure in Appendix D of When Good Disks Go Bad.