LVM failure; System Panic

[rianui]

Hi gurus,

I have an rx2600 system running HP-UX B11.23. After a power blackout the system failed to boot. When I accessed the console I found this error:

Loading.: HP-UX Primary Boot: 0/1/1/0.0.0
Load of HP-UX Primary Boot: 0/1/1/0.0.0 failed: Not Found
Paused - press any key to continue

I run the hpux command from fs0> and while booting I saw the following error messages:

Logical volume 64, 0x3 configured as ROOT
LVM : Failure in attaching PV (0/1/1/0.1.0) to the root volume group.
Cross device link. The disk is not a LVM disk.
LVM : Activation of root volume group failed
Quorum not present, or some physical volume(s) are missing

I then loaded the OS in maintenance mode from this point fs0:\EFI\HPUX>hpux -lm

vgdisplay gives me the following output:

# vgdisplay -v /dev/vg00
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "/dev/vg00".

Based on several comments I can see on the community, I have tried to run vgchange but still get the following error.

# vgchange -a y /dev/vg00
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c2t1d0s2":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c2t1d0s2":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "/dev/vg00":
Quorum not present, or some physical volume(s) are missing.

Kindly give me some pointers to get it running again.

Regards,

Rianui

[Torsten.]

Do you have mirrored disks?

You should try to boot from the alternate disk.

Most likely the boot headers got damaged, so without an alternate disks you may consider to use the recovery console or restore your ignite backup.

[rianui]

Hi Torsten,

I must thank you for such a fast response :)

This is where it gets tricky. The client does not know what I mean by ignite backup and has never seen this before. In short, it does not exist...plus all his 'backup' is on the same machine.

I have requested for a new drive and here is my plan:

- Install a new OS on the new drive and try to 'repair' the LVM. I'm still trying to figure out how I'll do that...

Regards,

Rianui

[Torsten.]

You can also boot from the appropriate install DVD, launch the recovery shell and try to repair from there.

Disks are not mirrored?

[rianui]

Hi Torsten,

I'd say the root disks are mirrored. lvdisplay however is not giving the requested info.

# lvdisplay /dev/vg00/lvol3

lvdisplay: Couldn't query logical volume "/dev/vg00/lvol3":
Volume group not activated.

# ioscan -fnCdisk

Class     I  H/W Path       Driver     S/W State   H/W Type     Description
============================================================================
disk      0  0/0/2/0.0.0.0  sdisk      CLAIMED     DEVICE       TEAC    DV-28E-C
                           /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
disk      1  0/1/1/0.0.0    sdisk      CLAIMED     DEVICE       HP 73.4GMAS3735NC
                           /dev/dsk/c2t0d0     /dev/rdsk/c2t0d0
                           /dev/dsk/c2t0d0s1   /dev/rdsk/c2t0d0s1
                           /dev/dsk/c2t0d0s2   /dev/rdsk/c2t0d0s2
                           /dev/dsk/c2t0d0s3   /dev/rdsk/c2t0d0s3
                           /dev/dsk/c2t0d0s4   /dev/rdsk/c2t0d0s4
                           /dev/dsk/c2t0d0s5   /dev/rdsk/c2t0d0s5
                           /dev/dsk/c2t0d0s6   /dev/rdsk/c2t0d0s6
                           /dev/dsk/c2t0d0s7   /dev/rdsk/c2t0d0s7
disk      2  0/1/1/0.1.0    sdisk      CLAIMED     DEVICE       HP 73.4GMAS3735NC
                           /dev/dsk/c2t1d0     /dev/rdsk/c2t1d0
                           /dev/dsk/c2t1d0s1   /dev/rdsk/c2t1d0s1
                           /dev/dsk/c2t1d0s2   /dev/rdsk/c2t1d0s2
                           /dev/dsk/c2t1d0s3   /dev/rdsk/c2t1d0s3
                           /dev/dsk/c2t1d0s4   /dev/rdsk/c2t1d0s4
                           /dev/dsk/c2t1d0s5   /dev/rdsk/c2t1d0s5
                           /dev/dsk/c2t1d0s6   /dev/rdsk/c2t1d0s6
                           /dev/dsk/c2t1d0s7   /dev/rdsk/c2t1d0s7

[Torsten.]

Can you boot from the other disk???

What happens if you boot in quorum mode?

You are in maintenance mode now?


Try

# vgchange -a y -q n vg00

[rianui]

Hi Torsten,

I tried that before. See the output below.

# vgchange -a y -q n vg00
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c2t1d0s2":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c2t1d0s2":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "vg00":
Quorum not present, or some physical volume(s) are missing.

[Torsten.]

But you booted from this disk?
In maintenance mode?


There is something really strange - usually there is s1, s2 and s3 - but you have s1 ...s7 - 7 partitions???



Did you boot from the other disk too??

[rianui]

Hi Torsten,

I am trying to boot from the alternate disk.

My map table looks like this:

 fs0  : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part1,Sig8A3D797A-FB9B-11D8-8002-D6217B60E588)
  fs1  : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part3,Sig8A3D79AC-FB9B-11D8-8004-D6217B60E588)
  fs2  : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun1,Lun0)/HD(Part1,SigDCA80A9C-FF80-11D9-8002-D6217B60E588)
  fs3  : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun1,Lun0)/HD(Part3,SigDCA80AD8-FF80-11D9-8004-D6217B60E588)
  blk0 : Acpi(HWP0002,0)/Pci(2|0)/Ata(Primary,Master)
  blk1 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)
  blk2 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part1,Sig8A3D797A-FB9B-11D8-8002-D6217B60E588)
  blk3 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part2,Sig8A3D7998-FB9B-11D8-8003-D6217B60E588)
  blk4 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part3,Sig8A3D79AC-FB9B-11D8-8004-D6217B60E588)
  blk5 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun1,Lun0)

[rianui]

fs0:\EFI\HPUX> hpux -lq

The command gives me the following output:

panic: LVM: Configuration failure
Stack Trace:
  IP                  Function Name
  0xe000000000e32300  lv_boothalt+0x110
  0xe000000000e31d70  lv_bootconf+0x3c0
  0xe000000000e3ad00  lvmconf+0x2e0
  0xe000000000e40930  im_lvmconf+0x70
  0xe0000000011d27d0  DoCalllist+0x3a0
End of Stack Trace

linkstamp:          Wed Sep 01 00:29:29 METDST 2004
_release_version:   @(#) $Revision: vmunix:    B11.23_LR FLAVOR=perf Fri Aug 29 22:35:38 PDT 2003 $

sync'ing disks (0 buffers to flush): (0 buffers to flush):
0 buffers not flushed
0 buffers still dirty
crash_callback: calling function e000000000d4da60

*** A system crash has occurred. (See the above messages for details.)
*** The system is now preparing to dump physical memory to disk, for use
*** in debugging the crash.

ERROR:  Your system crashed before I/O and dump configuration was complete.
        This system does not support a crash dump under these circumstances.
        Contact your HP support representative for assistance.