file system check failure, aborting ...

[sreedhar2u]

I have file systems attached to storage (DAS) to my HP-UX11i system. it is not mount the fie systems after root of the server. Please help me in addressing this issue.

Server1{HP-UX-root}# mountall
mountall: /dev/vg01/lvol11 has to be fsck'd
mountall: diagnostics from fsck
vxfs fsck: Cannot open /dev/vg01/lvol11: No such device or address
mountall: cannot fsck /dev/vg01/lvol11
mountall: diagnostics from fsck
file system check failure, aborting ...
No such device or address
mountall: /dev/vg01/lvol11 failed in fsck
mountall: /dev/vg01/lvol11 cannot be mounted
mountall: /dev/vg01/lvol13 has to be fsck'd
mountall: diagnostics from fsck
vxfs fsck: Cannot open /dev/vg01/lvol13: No such device or address
mountall: cannot fsck /dev/vg01/lvol13
mountall: diagnostics from fsck
file system check failure, aborting ...
No such device or address
mountall: /dev/vg01/lvol13 failed in fsck
mountall: /dev/vg01/lvol13 cannot be mounted
mountall: /dev/vg01/lvol15 has to be fsck'd
mountall: diagnostics from fsck
vxfs fsck: Cannot open /dev/vg01/lvol15: No such device or address
mountall: cannot fsck /dev/vg01/lvol15
mountall: diagnostics from fsck
file system check failure, aborting ...
No such device or address
mountall: /dev/vg01/lvol15 failed in fsck
mountall: /dev/vg01/lvol15 cannot be mounted
mountall: /dev/vg01/lvol14 has to be fsck'd
mountall: diagnostics from fsck
vxfs fsck: Cannot open /dev/vg01/lvol14: No such device or address
mountall: cannot fsck /dev/vg01/lvol14
mountall: diagnostics from fsck
file system check failure, aborting ...
No such device or address
mountall: /dev/vg01/lvol14 failed in fsck
mountall: /dev/vg01/lvol14 cannot be mounted

Regards,
Sreedhar

[Torsten.]

>> Cannot open /dev/vg01/lvol11: No such device or address


First do a

# vgdisplay -v vg01

[sreedhar2u]

I am getting the below

{HP-UX-root}# vgdisplay -v vg01
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "vg01".

[Torsten.]

Try

# vgchange -a y vg01

If in cluster, check from other nodes.

[sreedhar2u]

It is giving this error

{HP-UX-root}# vgchange -a y vg01
vgchange: Couldn't activate volume group "vg01":
Quorum not present, or some physical volume(s) are missing.

[Torsten.]

OK, a missing (dead) disk.

Try

# vgchange -a y -q n vg01

# vgdisplay -v vg01

[sreedhar2u]

still the same result

{HP-UX-root}# vgchange -a y -q n vg01
vgchange: Couldn't activate volume group "vg01":
Quorum not present, or some physical volume(s) are missing.

[Torsten.]

Let's try something different:

# strings /etc/lvmtab

# ioscan -fn

[sreedhar2u]

{HP-UX-root}# strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2t0d0
/dev/vg01
/dev/dsk/c5t0d0
/dev/dsk/c5t2d0
/dev/dsk/c5t4d0
/dev/vg02
/dev/dsk/c2t1d0

{HP-UX-root}# ioscan -fn
Class I H/W Path Driver S/W State H/W Type Description
=============================================================================
root 0 root CLAIMED BUS_NEXUS
ioa 0 0 sba CLAIMED BUS_NEXUS System Bus Adapter (880)
ba 0 0/0 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
usb 0 0/0/1/0 hcd CLAIMED INTERFACE NEC USB Interface
usbhub 0 0/0/1/0.1 hub CLAIMED DEVICE USB Root Hub
usb 1 0/0/1/1 hcd CLAIMED INTERFACE NEC USB Interface
usbhub 1 0/0/1/1.1 hub CLAIMED DEVICE USB Root Hub
/dev/hub
usb 2 0/0/1/2 ehci CLAIMED INTERFACE NEC USB Interface
sideba 0 0/0/2/0 side_ba CLAIMED INTERFACE CMD IDE controller
ext_bus 0 0/0/2/0.0 side CLAIMED INTERFACE IDE
target 0 0/0/2/0.0.0 tgt CLAIMED DEVICE
disk 0 0/0/2/0.0.0.0 sdisk CLAIMED DEVICE TEAC DV-28E-C
/dev/dsk/c0t0d0 /dev/rdsk/c0t0d0
target 1 0/0/2/0.0.7 tgt CLAIMED DEVICE
ctl 0 0/0/2/0.0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c0t7d0
ext_bus 1 0/0/2/0.1 side CLAIMED INTERFACE IDE
target 2 0/0/2/0.1.7 tgt CLAIMED DEVICE
ctl 1 0/0/2/0.1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c1t7d0
ba 1 0/1 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
ext_bus 2 0/1/1/0 c8xx CLAIMED INTERFACE SCSI C1010 Ultra160 Wide LVD
target 3 0/1/1/0.0 tgt CLAIMED DEVICE
disk 1 0/1/1/0.0.0 sdisk CLAIMED DEVICE HP 36.4GST336753LC
/dev/dsk/c2t0d0 /dev/rdsk/c2t0d0
target 4 0/1/1/0.1 tgt CLAIMED DEVICE
disk 6 0/1/1/0.1.0 sdisk CLAIMED DEVICE HP 146 GST3146707LC
/dev/dsk/c2t1d0 /dev/rdsk/c2t1d0
target 5 0/1/1/0.7 tgt CLAIMED DEVICE
ctl 2 0/1/1/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c2t7d0
ext_bus 3 0/1/1/1 c8xx CLAIMED INTERFACE SCSI C1010 Ultra160 Wide LVD
target 6 0/1/1/1.7 tgt CLAIMED DEVICE
ctl 3 0/1/1/1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c3t7d0
lan 0 0/1/2/0 igelan CLAIMED INTERFACE HP PCI 1000Base-T Core
ba 2 0/2 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
ba 3 0/3 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
ext_bus 5 0/3/1/0 c8xx CLAIMED INTERFACE SCSI C1010 Ultra160 Wide LVD A6828-60101
target 7 0/3/1/0.0 tgt CLAIMED DEVICE
disk 2 0/3/1/0.0.0 sdisk CLAIMED DEVICE HP 36.4GST336607LC
/dev/dsk/c5t0d0 /dev/rdsk/c5t0d0
target 8 0/3/1/0.2 tgt CLAIMED DEVICE
disk 3 0/3/1/0.2.0 sdisk CLAIMED DEVICE HP 36.4GST336607LC
/dev/dsk/c5t2d0 /dev/rdsk/c5t2d0
target 9 0/3/1/0.4 tgt CLAIMED DEVICE
disk 4 0/3/1/0.4.0 sdisk CLAIMED DEVICE HP 36.4GMAP3367NC
/dev/dsk/c5t4d0 /dev/rdsk/c5t4d0
target 10 0/3/1/0.7 tgt CLAIMED DEVICE
ctl 5 0/3/1/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c5t7d0
ba 4 0/4 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
ba 5 0/6 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
ba 6 0/7 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (783)
tty 0 0/7/1/0 asio0 CLAIMED INTERFACE PCI SimpleComm (103c1290)
/dev/diag/mux0 /dev/mux0 /dev/tty0p0
tty 1 0/7/1/1 asio0 CLAIMED INTERFACE PCI Serial (103c1048)
/dev/GSPdiag1 /dev/mux1 /dev/tty1p2
/dev/diag/mux1 /dev/tty1p0 /dev/tty1p4
unknown -1 0/7/2/0 UNCLAIMED UNKNOWN PCI Display (10025159)
memory 0 8 memory CLAIMED MEMORY Memory
ipmi 0 16 ipmi CLAIMED INTERFACE IPMI Controller
/dev/ipmi
processor 0 128 processor CLAIMED PROCESSOR Processor
processor 1 129 processor CLAIMED PROCESSOR Processor

[Torsten.]

Disks are there, try

# diskinfo /dev/rdsk/c5t0d0
# diskinfo /dev/rdsk/c5t2d0
# diskinfo /dev/rdsk/c5t4d0

[sreedhar2u]

{HP-UX-root}# diskinfo /dev/rdsk/c5t0d0
SCSI describe of /dev/rdsk/c5t0d0:
vendor: HP 36.4G
product id: ST336607LC
type: direct access
size: 35566480 Kbytes
bytes per sector: 512

{HP-UX-root}# diskinfo /dev/rdsk/c5t2d0
SCSI describe of /dev/rdsk/c5t2d0:
vendor: HP 36.4G
product id: ST336607LC
type: direct access
size: 35566480 Kbytes
bytes per sector: 512

{HP-UX-root}# diskinfo /dev/rdsk/c5t4d0
SCSI describe of /dev/rdsk/c5t4d0:
vendor: HP 36.4G
product id: MAP3367NC
type: direct access
size: 35566480 Kbytes
bytes per sector: 512

[Bill Hassell · Accepted solution]

> vxfs fsck: Cannot open /dev/vg01/lvol11: No such device or address

The lvol does not exist. It will not exist if the volume is not activated.

> vgdisplay: Volume group not activated.

This is why fsck is failing.

> vgchange: Couldn't activate volume group "vg01":
> Quorum not present, or some physical volume(s) are missing.

The volume cannot be activated since required disks are not found. They are found by reading the LVM header on the disk. Since ioscan reports the disks are present and diskinfo can query the disks, then something has changed the header information. You can use the attached script showLVMinfo to report back what is on each disk:

# showLVMinfo c5t0d0 c5t2d0 c5t4d0

It will either report a valid LVMREC for each disk, or report back that there no LVMREC at all, or report on some other volume group serial numbers. Post the results from the above script.

[sreedhar2u]

it is giving syntax error

{HP-UX-root}# showLVMinfo c5t0d0 c5t2d0 c5t4d0
showLVMinfo[21]: Syntax error at line 29 : `do' is not expected.

[宋明哲]

that's means Ur Three disk missing the VGRA; Have U Pvcreate these disk from this Server or Other server?

[sreedhar2u]

from the same server

[stephen peng]

you have pvcreate one or all pvs of that vg01? oh, you would miss vg01.

[Torsten.]

You did a pvcreate ???

[Bill Hassell]

> Syntax error at line 29 : `do' is not expected.

When you transferred the file to the HP-UX box, did you specify ASCII and not BINARY?

[sreedhar2u]

Oh, script working now

# ./showLVMinfo c5t0d0 c5t2d0 c5t4d0

Local CPU ID = 1761581075, hex value=0x68ff9813

/dev/rdsk/c5t0d0:
CPUID=0x68ff9813 (1761581075), VGID 0x4194a5e6, PVID 0x4194a5cb

/dev/rdsk/c5t2d0:
CPUID=0x68ff9813 (1761581075), VGID 0x4194a5e6, PVID 0x4194a5d1

/dev/rdsk/c5t4d0:
CPUID=0x68ff9813 (1761581075), VGID 0x4194a5e6, PVID 0x4194a5d7

[sudheerch]

#pvdisplay -v /dev/rdsk/c5t0d0
#pvdisplay -v /dev/rdsk/c5t2d0
#pvdisplay -v /dev/rdsk/c5t4d0

[sreedhar2u]

# pvdisplay -v /dev/dsk/c5t0d0
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c5t0d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c5t2d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c5t4d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query all of the physical volumes.
pvdisplay: Couldn't retrieve the names of the physical volumes
belonging to volume group "/dev/vg01".
pvdisplay: Cannot display physical volume "/dev/dsk/c5t0d0".

[Bill Hassell]

> Local CPU ID = 1761581075, hex value=0x68ff9813

So your computer ID is 0x68ff9813 (in hex).

> /dev/rdsk/c5t0d0:
CPUID=0x68ff9813 (1761581075), VGID 0x4194a5e6, PVID 0x4194a5cb

So these 3 disks do belong to your computer and they are all part of the same volume group (VGID=0x4194a5e6). And you can read the LVM header with no problems. Run this command:

# strings /etc/lvmtab

I suspect that vg01 has different disks listed in lvmtab. The fix will be to:

# ll /dev/vg01/group
(this will show you the current group minor number, probably 0x010000)

# vgexport -s -m vg01.map vg01
# mkdir /dev/vg01
# mknod /dev/vg01/group c 64 0x010000
(0x010000 is from the previous ll command)

# vgimport -s -m vg01.map vg01
# vgchange -a y vg01

Now you should be able to run: mount -a
fsck as necessary.

[sreedhar2u]

I have files in /dev/vg01. will it create any problem if run the above commands?

# cd /dev/vg01
# ls -lrt
total 0
cr--r--r-- 1 root sys 64 0x010000 Nov 12 2004 group
br--r----- 1 root sys 64 0x010001 Nov 15 2004 lvol11
cr--r----- 1 root sys 64 0x010001 Nov 15 2004 rlvol11
cr--r----- 1 root sys 64 0x010003 Nov 23 2004 rlvol13
br--r----- 1 root sys 64 0x010003 Nov 23 2004 lvol13
br--r----- 1 root sys 64 0x010004 Nov 23 2004 lvol14
cr--r----- 1 root sys 64 0x010004 Nov 23 2004 rlvol14
br--r----- 1 root sys 64 0x010005 Nov 23 2004 lvol15
cr--r----- 1 root sys 64 0x010005 Nov 23 2004 rlvol15

[Bill Hassell]

Those files are pointing to non-existent disks. It would be better to run this command and post the results:

# strings /etc/lvmtab

vgexport will remove the directory /dev/vg01 and all the files. These are device files and they no longer work correctly. vgimport (as part of the above commands) will recreate these files and connect them with the correct disks.

[Torsten.]

@Bill:

Its posted earlier in this thread:

HP-UX-root}# strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2t0d0
/dev/vg01
/dev/dsk/c5t0d0
/dev/dsk/c5t2d0
/dev/dsk/c5t4d0
/dev/vg02
/dev/dsk/c2t1d0