vmunix: SCSI Read Error/Async Write Error

[Christine S Feeley]

This problem has occurred three times in the past two days. I am using secure path. Is this a path problem or a disk going bad? The virtual array shows all disks and LUNs as good .

Apr 2 16:12:43 rgitip95 vmunix: SCSI: Read error -- dev: b 31 0x060000, errno:
126, resid: 1024,
Apr 2 16:12:43 rgitip95 vmunix: blkno: 8, sectno: 16, offset: 8192, bcou
nt: 1024.
Apr 2 16:12:47 rgitip95 vmunix: SCSI: Read error -- dev: b 31 0x063100, errno:
126, resid: 1024,
Apr 2 16:12:50 rgitip95 vmunix: SCSI: Read error -- dev: b 31 0x066400, errno:
126, resid: 1024,
Apr 2 16:12:50 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e000000134a1c000), from raw device 0x1f066400 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f086400 (with priority: 1, and current flags: 0x0).
Apr 2 16:12:50 rgitip95 vmunix:
Apr 2 16:12:50 rgitip95 above message repeats 2 times
Apr 2 16:12:50 rgitip95 vmunix: LVM: VG 64 0x60000: PVLink 31 0x66400 Failed!.
The PV is still accessible.
Apr 2 16:12:50 rgitip95 vmunix: LVM: VG 64 0x60000: PVLink 31 0x66500 Failed!.
The PV is not accessible.
Apr 2 16:12:50 rgitip95 vmunix: blkno: 8, sectno: 16, offset: 8192, bcou
nt: 1024.
Apr 2 16:12:52 rgitip95 above message repeats 2 times
Apr 2 16:12:52 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e000000134e5c000), from raw device 0x1f066500 (with priority: 0, and current fla
gs: 0xc0) to raw device 0x1f086500 (with priority: 1, and current flags: 0x0).
e00000013475e000), from raw device 0x1f063100 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f083100 (with priority: 1, and current flags: 0x0).
Apr 2 16:12:56 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e0000001347be000), from raw device 0x1f063300 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f083300 (with priority: 1, and current flags: 0x0).
Apr 2 16:12:56 rgitip95 vmunix: LVM: VG 64 0x30000: PVLink 31 0x63100 Failed!.
The PV is still accessible.
Apr 2 16:12:56 rgitip95 vmunix: LVM: VG 64 0x30000: PVLink 31 0x63300 Failed!.
The PV is still accessible.
Apr 2 16:13:01 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013451e000), from raw device 0x1f060000 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f080000 (with priority: 1, and current flags: 0x0).
Apr 2 16:13:01 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e0000001345d6000), from raw device 0x1f061000 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f081000 (with priority: 1, and current flags: 0x0).
Apr 2 16:13:01 rgitip95 vmunix: LVM: VG 64 0x20000: PVLink 31 0x60000 Failed!.
The PV is still accessible.
Apr 2 16:13:01 rgitip95 vmunix: LVM: VG 64 0x20000: PVLink 31 0x61000 Failed!.
The PV is still accessible.
Apr 2 16:13:02 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013475e000), from raw device 0x1f083100 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f063100 (with priority: 0, and current flags: 0x80).
Apr 2 16:13:02 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
gs: 0x0) to raw device 0x1f063300 (with priority: 0, and current flags: 0x80).
Apr 2 16:13:02 rgitip95 vmunix: LVM: VG 64 0x30000: PVLink 31 0x63100 Recovered
.
Apr 2 16:13:02 rgitip95 vmunix: LVM: VG 64 0x30000: PVLink 31 0x63300 Recovered
.
Apr 2 16:13:02 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e000000134e5c000), from raw device 0x1f086500 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f066500 (with priority: 0, and current flags: 0x80).
Apr 2 16:13:02 rgitip95 vmunix: LVM: VG 64 0x60000: PVLink 31 0x66400 Recovered
.
Apr 2 16:13:02 rgitip95 vmunix: LVM: VG 64 0x60000: PVLink 31 0x66500 Recovered
.
Apr 2 16:13:02 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e000000134a1c000), from raw device 0x1f086400 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f066400 (with priority: 0, and current flags: 0x0).
Apr 2 16:13:06 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013451e000), from raw device 0x1f080000 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f060000 (with priority: 0, and current flags: 0x80).
Apr 2 16:13:06 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e0000001345d6000), from raw device 0x1f081000 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f061000 (with priority: 0, and current flags: 0x80).
Apr 2 16:13:06 rgitip95 vmunix: LVM: VG 64 0x20000: PVLink 31 0x60000 Recovered
.
.
Apr 2 17:12:30 rgitip95 LVM[12252]: /sbin/lvlnboot -v vg00
Apr 2 17:12:30 rgitip95 LVM[12255]: /sbin/lvlnboot -v vg00
Apr 2 17:12:31 rgitip95 LVM[12261]: lvlnboot -v
Apr 2 17:12:33 rgitip95 LVM[12278]: /sbin/lvlnboot -v vg00
Apr 2 17:12:33 rgitip95 LVM[12281]: /sbin/lvlnboot -v vg00
Apr 2 17:25:26 rgitip95 vmunix:
Apr 2 17:25:26 rgitip95 vmunix: SCSI: Read error -- dev: b 31 0x060000, errno:
126, resid: 1024,
Apr 2 17:25:26 rgitip95 vmunix: blkno: 8, sectno: 16, offset: 8192, bcou
nt: 1024.
Apr 2 17:25:27 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013451e000), from raw device 0x1f060000 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f080000 (with priority: 1, and current flags: 0x0).
Apr 2 17:25:27 rgitip95 vmunix: LVM: VG 64 0x20000: PVLink 31 0x60000 Failed!.
The PV is still accessible.
Apr 2 17:25:32 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013451e000), from raw device 0x1f080000 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f060000 (with priority: 0, and current flags: 0x80).
Apr 2 17:25:32 rgitip95 vmunix: LVM: VG 64 0x20000: PVLink 31 0x60000 Recovered
.
Apr 2 17:29:26 rgitip95 vmunix:
Apr 2 17:29:26 rgitip95 vmunix: SCSI: Read error -- dev: b 31 0x060100, errno:
126, resid: 1024,
Apr 2 17:29:26 rgitip95 vmunix: blkno: 8, sectno: 16, offset: 8192, bcou
nt: 1024.
Apr 2 17:29:26 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013437c000), from raw device 0x1f060100 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f080100 (with priority: 1, and current flags: 0x0).
Apr 2 17:29:26 rgitip95 vmunix: LVM: VG 64 0x10000: PVLink 31 0x60100 Failed!.
The PV is still accessible.
Apr 2 17:29:34 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013437c000), from raw device 0x1f080100 (with priority: 1, and current fla
gs: 0x0) to raw device 0x1f060100 (with priority: 0, and current flags: 0x80).
Apr 2 17:29:34 rgitip95 vmunix: LVM: VG 64 0x10000: PVLink 31 0x60100 Recovered
.
Apr 2 17:42:11 rgitip95 vmunix:
Apr 2 17:42:11 rgitip95 vmunix: SCSI: Read error -- dev: b 31 0x060100, errno:
126, resid: 1024,
Apr 2 17:42:11 rgitip95 vmunix: blkno: 8, sectno: 16, offset: 8192, bcou
nt: 1024.
Apr 2 17:42:13 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
e00000013437c000), from raw device 0x1f060100 (with priority: 0, and current fla
gs: 0x40) to raw device 0x1f080100 (with priority: 1, and current flags: 0x0).
Apr 2 17:42:13 rgitip95 vmunix: LVM: VG 64 0x10000: PVLink 31 0x60100 Failed!.
The PV is still accessible.
Apr 2 17:42:18 rgitip95 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x
gs: 0x0) to raw device 0x1f060100 (with priority: 0, and current flags: 0x80).
Apr 2 17:42:18 rgitip95 vmunix: LVM: VG 64 0x10000: PVLink 31 0x60100 Recovered
.

[Steven E. Protter]

You had a scsi timeout of some kind. It could be triggered by very busy disks, bad cables, lots of possible causes.

If its happening a lot, you need to look into the root causes. You can simply increase the timeout on your disks. That almost never works.

If you had two paths to your disks, its possible one of them failed.

dmesg

Will help you find hardware problems.

cstm mstm xstm are good tools for testing hardware.

SEP

[Jim Butler]

Christine
You should probably call HP and discuss with them - It could be simply that you have added a load, and if you increase the timeout length on the controller, then these messages will go away; however, there could be something more problematic happening.
Good Luck

[Christine S Feeley]

I called HP yesterday, and sent them several files. I applied phkl_31501 for Fiber Channel Mass Storage and it seems to have cleared the problems. What I don't understand is how this patch would all of a sudden become an issue with this server that has been running fine for over a year without any changes? I'll probably never understand that one.

Thanks to everyone for you feedback...the issue has been resolved.

[Christine S Feeley]

See above posting