Problem with disk drive in syslog.log

[james gould]

Getting error on drive but disk array not producing any errors after running armdsp -a on the array. Oracle RMAN also is having problems backing up. Any help is appreciated.

rp4640
Hpux 11.11
Va7110 - connected to fibre switch

Syslog.log output below:

vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x0000
000049d2b040), from raw device 0x1f080100 (with priority: 0, and current flags:
0x40) to raw device 0x1f0a0100 (with priority: 1, and current flags: 0x0).
Jul 28 11:14:32 vmunix:
Jul 28 11:14:32 vmunix: SCSI: Read error -- dev: b 31 0x080100, errno: 126,
resid: 2048,
Jul 28 11:14:32 vmunix: blkno: 8, sectno: 16, offset: 8192, bcount: 2048
.
Jul 28 11:14:32 vmunix: LVM: vg[1]: pvnum=0 (dev_t=0x1f0a0100) is POWERFAIL
ED
Jul 28 11:14:40 vmunix: LVM: Recovered Path (device 0x1f080100) to PV 0 in
VG 1.
Jul 28 11:14:40 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x0000
000049d2b040), from raw device 0x1f0a0100 (with priority: 1, and current flags:
0xc0) to raw device 0x1f080100 (with priority: 0, and current flags: 0x80).
Jul 28 11:14:40 vmunix: LVM: Recovered Path (device 0x1f0a0100) to PV 0 in
VG 1.
Jul 28 11:14:32 vmunix: LVM: vg[1]: pvnum=0 (dev_t=0x1f0a0100) is POWERFAIL
ED
Jul 28 11:14:40 vmunix: LVM: Restored PV 0 to VG 1.

[A. Clay Stephenson]

What you are seeing is that /dev/dsk/c8t1d0 (a LUN) stopped responding to SCSI commands (errno 126 is rather a general purpose error) and switched to an alternate path on /dev/dsk/c10t1d0. The problem could actually lie anywhere along the SCSI/Fibre chain from the controller in the host, the switch, or the array itself. If the array diagnostics are reporting nothing then I would examine your switch for any diagnostics. I have seen controllers cause this problem but so can simply barely bad fibre cable (or simply coiled too tightly).

However, after looking at this more closely you had a failure on both your primary path and later on your alternate so that tends to point to a common failure point. I assume that the primary and alternate paths go through separate switches and ports on the array -- if that is a bad assumption then they should. If the paths are completely separate then this tends to point to some kind of failure in the array itself.

Though this probably isn't your problem, you should check the timeouts on your array LUN's using pvdisplay. The default (30 seconds) is designed for standalone drives but array LUN's should be set to something in the 90-150 second range using pvchange.

[skt_skt]

check with following command.

diskinfo
pvdisplay
ioscan

it would be better to know what type of SAN disk you have,

[Torsten.]

The type of array is mentioned in the question, it is a

VA7110.

The messages "POWERFAILED" does not mean the array has failed, but the server cannot reach the disk anymore.

Check the array status using armdsp -a first.
Is the status ready? Check the switch port statistics for errors.
Based on the information you gave us, it could be anything - a bad controller, a bad disk disturbing the internal loops, a bad cable, a bad GBIC ... more investigation needed.