PVLink 31 0x06a000 Failed!

[bill_allen]

I have received the following errors in my syslog.log. It started on May 13, and finally finished on May 17 with:
May 17 13:32:15 fcdshp03 vmunix: LVM: VG 64 0x010000: PVLink 31 0x06a000 Failed! The PV is not accessible.
May 17 13:32:15 fcdshp03 vmunix: LVM: VG 64 0x010000: PVLink 31 0x06a000 Recovered.

But no further errors in the log since then. What has happened or is happening? Why would the error suddenly resolve? The complete listing is below. Any information would be helpful and appreciated.

Bill
===================
May 17 13:32:11 fcdshp03 vmunix: SCSI: isrEscape Controller at 0/6/0/0.
May 17 13:32:11 fcdshp03 vmunix: SCSI: Parity error -- lbolt: 286419065, dev: 1f060000
May 17 13:32:11 fcdshp03 vmunix: lbp->state: 30008
May 17 13:32:11 fcdshp03 vmunix: lbp->offset: ffffffff
May 17 13:32:11 fcdshp03 vmunix:
May 17 13:32:11 fcdshp03 above message repeats 2 times
May 17 13:32:11 fcdshp03 vmunix: lbp->nominalOffset: 330
May 17 13:32:11 fcdshp03 vmunix: lbp->Cmdindex: 4
May 17 13:32:11 fcdshp03 vmunix: lbp->last_nexus_index: 1c
May 17 13:32:11 fcdshp03 vmunix: lbp->nexus_index: 1d
May 17 13:32:11 fcdshp03 vmunix: uCmdSent: 2005880 uNexus_offset: 25674
May 17 13:32:11 fcdshp03 vmunix: last lbp->puStatus [0000000040215654]:
May 17 13:32:11 fcdshp03 vmunix: ff031a00 ff031400 ff031200 00030062
May 17 13:32:11 fcdshp03 vmunix: next lbp->puStatus [0000000040215664]:
May 17 13:32:11 fcdshp03 vmunix: ff031200 ff031000 ff031400 00030069
May 17 13:32:11 fcdshp03 vmunix: From most recent interrupt:
May 17 13:32:11 fcdshp03 vmunix: ISTAT: 0a, SIST0: 91, SIST1: 00, DSTAT: 80, DSPS: 00000000
May 17 13:32:11 fcdshp03 vmunix: lsp: 0x0000000055bc0a00
May 17 13:32:11 fcdshp03 vmunix: bp->b_dev: 1f060000
May 17 13:32:11 fcdshp03 vmunix: scb->io_id: 65de581
May 17 13:32:11 fcdshp03 vmunix: scb->cdb: 28 00 00 2d a6 80 00 00 80 00
May 17 13:32:11 fcdshp03 vmunix: lbolt_at_timeout: 0, lbolt_at_start: 0
May 17 13:32:11 fcdshp03 vmunix: lsp->state: 4005
May 17 13:32:11 fcdshp03 vmunix: Jump Table entry [fffffffffb7fccf8]: ff01007c 00021040
May 17 13:32:11 fcdshp03 vmunix: lsp->puScript [0000000040211040]:
May 17 13:32:11 fcdshp03 vmunix: 09010000 00820000 00000000 78370000
May 17 13:32:11 fcdshp03 vmunix: 00000000 80080000 fb7fc288 00000000
May 17 13:32:11 fcdshp03 vmunix: DSAtbl->host_iocb_index: 4
May 17 13:32:11 fcdshp03 vmunix: DSAtbl->host_iocb_addr: 25840
May 17 13:32:11 fcdshp03 vmunix: stored scratcha: 0xff03007c
May 17 13:32:11 fcdshp03 vmunix: scratch_lsp: 0x0000000055bc0a00
May 17 13:32:11 fcdshp03 vmunix: c8xx_iocb [fffffffffb7fcb00]:
May 17 13:32:11 fcdshp03 vmunix: 02005880 ff005400 00025400 9f043e80
May 17 13:32:11 fcdshp03 vmunix: 00000009 00025860 00000006 0002586c
May 17 13:32:11 fcdshp03 vmunix: Pre-DSP script dump [0000000040211024]:
May 17 13:32:11 fcdshp03 vmunix: fb7fc288 00000000 00000000 00000000
May 17 13:32:11 fcdshp03 vmunix: 00000000 00000000 00000000 09010000
May 17 13:32:11 fcdshp03 vmunix: Script dump [0000000040211044]:
May 17 13:32:11 fcdshp03 vmunix: 00820000 00000000 78370000 00000000
May 17 13:32:11 fcdshp03 vmunix: 80080000 fb7fc288 00000000 80080000
May 17 13:32:11 fcdshp03 vmunix: NCR chip register dump for: 0x400200a
May 17 13:32:11 fcdshp03 vmunix: 00: SCNTL3: 9f SCNTL2: 80 SCNTL1: 10 SCNTL0: da
May 17 13:32:11 fcdshp03 vmunix: 04: GPREG: 0a SDID: 04 SXFER: 3e SCID: 47
May 17 13:32:11 fcdshp03 vmunix: 08: SBCL: ae SSID: 80 SOCL: 0d SFBR: 01
May 17 13:32:11 fcdshp03 vmunix: 0c: SSTAT2: 09 SSTAT1: 06 SSTAT0: 01 DSTAT: 80
May 17 13:32:11 fcdshp03 vmunix: 10: DSA: fb7fcb00
May 17 13:32:11 fcdshp03 vmunix: 14: MBOX1: 00 MBOX0: 00 ISTAT1: 00 ISTAT: 08
May 17 13:32:11 fcdshp03 vmunix: 1c: TEMP: 00021040
May 17 13:32:11 fcdshp03 vmunix: 24: DCMDDBC: 0d00f800
May 17 13:32:11 fcdshp03 vmunix: 28: DNAD: 00820800
May 17 13:32:11 fcdshp03 vmunix: 2c: DSP: 0002104c
May 17 13:32:11 fcdshp03 vmunix: 30: DSPS: 00820000
May 17 13:32:11 fcdshp03 vmunix: 34: SCRATCHA: ff03007c
May 17 13:32:11 fcdshp03 vmunix: 38: DCNTL: a1 DWT: 00 DIEN: 7f DMODE: 4c
May 17 13:32:11 fcdshp03 vmunix: 3c: ADDER: 0084104c
May 17 13:32:11 fcdshp03 vmunix: 40: SIST1: 00 SIST0: 00 SIEN1: 97 SIEN0: 8f
May 17 13:32:11 fcdshp03 vmunix: 44: GPCNTL: 2f MACNTL: 00 SWIDE: 00 SLPAR: 00
May 17 13:32:11 fcdshp03 vmunix: 48: RESPID1: 00 RESPID0: 80 STIME1: 00 STIME0: fc
May 17 13:32:11 fcdshp03 vmunix: 4c: STEST3: 80 STEST2: 00 STEST1: 0c STEST0: 76
May 17 13:32:11 fcdshp03 vmunix: 50: RESV50: 00 RESV51: c0 SIDL1: 00 SIDL0: 7c
May 17 13:32:11 fcdshp03 vmunix: 54: CCNTL1: 01 CCNTL0: 01 SODL1: 00 SODL0: 00
May 17 13:32:11 fcdshp03 vmunix: 58: RESV58: 00 RESV59: 00 SBDL1: 00 SBDL0: 00
May 17 13:32:11 fcdshp03 vmunix: 5c: SCRATCHB: 00000013
May 17 13:32:11 fcdshp03 vmunix: 60: SCRATCHC: c0ffffff
May 17 13:32:11 fcdshp03 vmunix: 64: SCRATCHD: 00021040
May 17 13:32:11 fcdshp03 vmunix: 68: SCRATCHE: fb7fccfc
May 17 13:32:11 fcdshp03 vmunix: 6c: SCRATCHF: 00020400
May 17 13:32:11 fcdshp03 vmunix: 70: SCRATCHG: 9f043e80
May 17 13:32:11 fcdshp03 vmunix: 74: SCRATCHH: 00025674
May 17 13:32:11 fcdshp03 vmunix: 78: SCRATCHI: 09819f3e
May 17 13:32:11 fcdshp03 vmunix: 7c: SCRATCHJ: 01005880
May 17 13:32:11 fcdshp03 vmunix: bc: SCNTL4: 80
May 17 13:32:11 fcdshp03 vmunix: PCI configuration register dump:
May 17 13:32:11 fcdshp03 vmunix: Command: 0157
May 17 13:32:11 fcdshp03 vmunix: Latency Timer: ff
May 17 13:32:11 fcdshp03 vmunix: Cache Line Size: 10
May 17 13:32:12 fcdshp03 vmunix:
May 17 13:32:12 fcdshp03 vmunix: SCSI: Resetting SCSI -- lbolt: 286419165, bus: 6 path: 0/6/0/0
May 17 13:32:12 fcdshp03 vmunix: SCSI: Reset detected -- lbolt: 286419165, bus: 6 path: 0/6/0/0
May 17 13:32:15 fcdshp03 vmunix: LVM: VG 64 0x010000: PVLink 31 0x06a000 Failed! The PV is not accessible.
May 17 13:32:15 fcdshp03 vmunix: LVM: VG 64 0x010000: PVLink 31 0x06a000 Recovered.
May 17 13:32:12 fcdshp03 vmunix:

[Florian Heigl (new acc)]

I'll try to shorten out the log a bit:

13:32:11: Your scsi controller controller notices a parity error on a communication from the disk 1f060000
13:32:12 the controller reset the affected scsi bus to regather status quo (I think)
13:32:12 (a bit later) the controller acknoleged the reset was underway
13:32:12 during the few seconds the bus reset takes, LVM noticed it couldn't access the disk
13:32:15 after 3 seconds the reset was over, LVM could access the disk again.

As the parity mostly safeguards against data corruption, I'd be ok with a single error every now and then. if it were >15 consequitive errors, a hardware technician should have a look at the scsi-level error messages the controller supplied.

[Ninad_1 · Accepted solution]

The path to the disk seems to have been inaccessible for some moment. The SCSI bus was resetted.

The volume group the disk is in can be checked by
ls -l /dev/*/group | grep '0x010000'

The disk failed should be c6t10d0.
Usually these type of errors can be due to improper termination/ problem in the path to the disk. Keep an eye for any further errors and have a backup in place if not already.

Regards,
Ninad

[bill_allen]

Great information guys! I have forwarded the log info to my hardware support guys to make a determination if they need to come on site soon. Thanks again! :-)