Operating System - HP-UX
1748180 Members
4304 Online
108759 Solutions
New Discussion юеВ

Taming of the Shrew (Ghost Disk)

 
SOLVED
Go to solution
Ralph Grothe
Honored Contributor

Taming of the Shrew (Ghost Disk)

Hi,

I lost a mirror root/boot disk from vg00 on an ole' D-Class.
This must have upset the the system so much that it paniced and could hardly be convinced to reboot in normal runlevel even when selecting alternate boot device, and checking things in maintenace mode with no VGs activatet (e.g. it didn't want to mount /stand).
Anyway, I brought the system back to - so somewhat impaired - normal operation.
Now I encounter the usual trouble derriving from descrepencies between kernel and lvmtab meta data.
So when I got back from the console to my PC where I had access to the HP-UX Software Recovery Cookbook (i.e. Chpt. 16 LVM) I did so as outlined in the "Removing A Ghost Disk" section.
All the first hand hacks didn't help.
I was only to lvreduce the mirrors by the PV key hack (described therein).
lvdisplay isn't reporting the failed PV to be in the LVs anymore.

# lvdisplay -v /dev/vg00/lvol[1-9] 2>/dev/null|grep c1t0d0
#

The errors I swept under the carpet by redirection above all relate to the ghost disk and are repeatedly like this:

lvdisplay: Warning: couldn't query physical volume "/dev/dsk/c1t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvdisplay: Warning: couldn't query all of the physical volumes.

As there are no more LEs occupied from c1t0d0 even for LVM, thanks to the successful lvreduce, now I should be able (according to theory) to run "vgreduce -f vg00".
But this doesn't work, instead

# vgreduce -f vg00
vgreduce: Couldn't query physical volume "/dev/dsk/c1t0d0":
The specified path does not correspond to physical volume attached to
this volume group

I also tried in vain to have a new lvmtab created through vgscan.
But it won't get rid of c1t0d0.

I supplied a new disk for the failed one (hotswappable) and tried things like

pvcreate -f -B /dev/rdsk/c1t0d0

or

vgcfgrestore -n vg00 /dev/rdsk/c1t0d0

But these don't change a thing.

Any other hacks I could try?

Rgds.
Ralph
Madness, thy name is system administration
13 REPLIES 13
RAC_1
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Doing vgreduce -f vg00 should have taken care of it. What is the OS version??

what does the following command says?
strings /etc/lvmtab

Once again, do a lvdisplay on each of the lvs in vg00 and check if any of them list the faulty disk.

Anil
There is no substitute to HARDWORK
Ralph Grothe
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

RAC,

meanwhile I'm a step farther.
I had to completely mv or remove the /etc/lvmtab, and rerun vgscan.
This led to a new lvmtab with no c1t0d0 in vg00

# strings /etc/lvmtab|head
/dev/vg00
U8)A
/dev/dsk/c0t5d0
/dev/dsk/c0t8d0
/dev/dsk/c0t9d0
/dev/dsk/c2t0d0
/dev/vg01
/dev/dsk/c1t1d0
/dev/dsk/c2t1d0
/dev/vg02

However, I missed an important step which reconciles the mismatch between kernel and LVM, i.e.

# vgchange -a y vg00
Volume group "vg00" has been successfully changed.

No errors now

# vgdisplay vg00 >/dev/null

# vgdisplay -v vg00|grep PV\ Name
PV Name /dev/dsk/c0t5d0
PV Name /dev/dsk/c0t8d0
PV Name /dev/dsk/c0t9d0
PV Name /dev/dsk/c2t0d0


Now I have to reintegrate the PV again...
Madness, thy name is system administration
Ralph Grothe
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Hm, this annoyingly stubborn.
I still fail to succeed with the vgcfgbackup of vg00 :-(

# vgcfgbackup vg00
vgcfgbackup: /etc/lvmtab is out of date with the running kernel:Kernel indicates
5 disks for "/dev/vg00"; /etc/lvmtab has 4 disks.
Cannot proceed with backup.

Ok, force the replacement disk in...

# pvcreate -f -B /dev/rdsk/c1t0d0
Physical volume "/dev/rdsk/c1t0d0" has been successfully created.

# vgextend vg00 /dev/dsk/c1t0d0
Volume group "vg00" has been successfully extended.
vgcfgbackup: /etc/lvmtab is out of date with the running kernel:Kernel indicates
6 disks for "/dev/vg00"; /etc/lvmtab has 5 disks.
Cannot proceed with backup.

Madness, thy name is system administration
RAC_1
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

vgdisplay -v vg00

Check pvs in vg and active pvs.
Any mismatch there???

Anil
There is no substitute to HARDWORK
Ralph Grothe
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Though vg00 now contains the replacement disk again, I still fail to write the old LIF header on it.

# strings /etc/lvmtab|head
/dev/vg00
U8)A
/dev/dsk/c0t5d0
/dev/dsk/c0t8d0
/dev/dsk/c0t9d0
/dev/dsk/c2t0d0
/dev/dsk/c1t0d0
/dev/vg01
/dev/dsk/c1t1d0
/dev/dsk/c2t1d0

# vgdisplay -v vg00|grep PV\ Name
PV Name /dev/dsk/c0t5d0
PV Name /dev/dsk/c0t8d0
PV Name /dev/dsk/c0t9d0
PV Name /dev/dsk/c2t0d0
PV Name /dev/dsk/c1t0d0

# vgcfgrestore -n vg00 /dev/rdsk/c1t0d0
vgcfgrestore: Mismatch between the backup file and the running kernel:
Kernel indicates 6 disks for "/dev/vg00"; /etc/lvmconf/vg00.conf indicates 5 dis
ks.
Cannot proceed with the restoration. Deactivate the Volume Group and try again.
Madness, thy name is system administration
Ralph Grothe
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Yes RAC, the mismatch is still prevelant

# vgdisplay vg00|grep -e Cur\ PV -e Act\ PV
Cur PV 6
Act PV 5
Madness, thy name is system administration
Ralph Grothe
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Strange mess now,
lvmtab and vgdisplay are in sync

# strings /etc/lvmtab|sed -n /vg00/,/vg01/p
/dev/vg00
U8)A
/dev/dsk/c0t5d0
/dev/dsk/c0t8d0
/dev/dsk/c0t9d0
/dev/dsk/c2t0d0
/dev/dsk/c1t0d0
/dev/vg01

# vgdisplay -v vg00|grep PV\ Name
PV Name /dev/dsk/c0t5d0
PV Name /dev/dsk/c0t8d0
PV Name /dev/dsk/c0t9d0
PV Name /dev/dsk/c2t0d0
PV Name /dev/dsk/c1t0d0


How can I force kernel structures into acknowledgement of new reality?
Madness, thy name is system administration
Ralph Grothe
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Since I kicked off users anyway,
would a reboot resolve this mismatch?
Madness, thy name is system administration
RAC_1
Honored Contributor

Re: Taming of the Shrew (Ghost Disk)

Seems that it is reboot time now. As said earlier, this should have been OK with lvreduce (for faulty disk), vgreduce.

Are you up to date on LVM patches??

Reboot

Anil
There is no substitute to HARDWORK