scsi async error

[brian_31]

here is what i got from syslog
syslog.log:Feb 22 05:38:24 ag454 vmunix: SCSI: Async write error -- dev: b 31 0x010000, errno: 5, resid: 8192,

the resid changes but there are lot of errors ..what is this..pl help no NO_HW from ioscan

Thanks

brian

[brian_31]

# lvdisplay -v /dev/vg02/lvol1|more
--- Logical volumes ---
LV Name /dev/vg02/lvol1
VG Name /dev/vg02
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 7900
Current LE 1975
Allocated PE 3950
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c1t0d0 1975 1975
/dev/dsk/c2t0d0 1975 1975

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c1t0d0 00000 stale /dev/dsk/c2t0d0 00000 current
00001 /dev/dsk/c1t0d0 00001 stale /dev/dsk/c2t0d0 00001 current
00002 /dev/dsk/c1t0d0 00002 current /dev/dsk/c2t0d0 00002 current
00003 /dev/dsk/c1t0d0 00003 current /dev/dsk/c2t0d0 00003 current
00004 /dev/dsk/c1t0d0 00004 current /dev/dsk/c2t0d0 00004 current
00005 /dev/dsk/c1t0d0 00005 current /dev/dsk/c2t0d0 00005 current
00006 /dev/dsk/c1t0d0 00006 current /dev/dsk/c2t0d0 00006 current
00007 /dev/dsk/c1t0d0 00007 current /dev/dsk/c2t0d0 00007 current
00008 /dev/dsk/c1t0d0 00008 current /dev/dsk/c2t0d0 00008 current
00009 /dev/dsk/c1t0d0 00009 current /dev/dsk/c2t0d0 00009 current
00010 /dev/dsk/c1t0d0 00010 current /dev/dsk/c2t0d0 00010 current
00011 /dev/dsk/c1t0d0 00011 current /dev/dsk/c2t0d0 00011 current
00012 /dev/dsk/c1t0d0 00012 current /dev/dsk/c2t0d0 00012 current
00013 /dev/dsk/c1t0d0 00013 current /dev/dsk/c2t0d0 00013 current
00014 /dev/dsk/c1t0d0 00014 current /dev/dsk/c2t0d0 00014 current
00015 /dev/dsk/c1t0d0 00015 current /dev/dsk/c2t0d0 00015 current
00016 /dev/dsk/c1t0d0 00016 current /dev/dsk/c2t0d0 00016 current
00017 /dev/dsk/c1t0d0 00017 stale /dev/dsk/c2t0d0 00017 current
00018 /dev/dsk/c1t0d0 00018 current /dev/dsk/c2t0d0 00018 current
00019 /dev/dsk/c1t0d0 00019 current /dev/dsk/c2t0d0 00019 current
00020 /dev/dsk/c1t0d0 00020 current /dev/dsk/c2t0d0 00020 current
00021 /dev/dsk/c1t0d0 00021 current /dev/dsk/c2t0d0 00021 current
00022 /dev/dsk/c1t0d0 00022 current /dev/dsk/c2t0d0 00022 current
00023 /dev/dsk/c1t0d0 00023 current /dev/dsk/c2t0d0 00023 current
00024 /dev/dsk/c1t0d0 00024 current /dev/dsk/c2t0d0 00024 current
00025 /dev/dsk/c1t0d0 00025 current /dev/dsk/c2t0d0 00025 current
00026 /dev/dsk/c1t0d0 00026 current /dev/dsk/c2t0d0 00026 current
00027 /dev/dsk/c1t0d0 00027 current /dev/dsk/c2t0d0 00027 current
00028 /dev/dsk/c1t0d0 00028 current /dev/dsk/c2t0d0 00028 current
00029 /dev/dsk/c1t0d0 00029 current /dev/dsk/c2t0d0 00029 current
00030 /dev/dsk/c1t0d0 00030 current /dev/dsk/c2t0d0 00030 current

[A. Clay Stephenson]

Try a vgsync /dev/vg02; this may fix you up for now but your symptoms are indicative of a failing disk (c1t0d0) and you should replace it very soon.

[Victor BERRIDGE]

Hi Brian,
Do a vgdisplay of that group, seems you have a disk thats going....


All the best
Victor

[brian_31]

Can i try a vgsync with the apps running?

Thanks

brian

[Victor BERRIDGE]

I do...

[A. Clay Stephenson · Accepted solution]

Yes.

[Victor BERRIDGE]

Brian,
Since we suspect a disk may be failing, look at diskinfo of c1t0d0
Try also:
dd if=/dev/rdsk/c1t0d0 of=/dev/null bs=1024k
and see if you have I/O errors

Good luck
Victor

[brian_31]

Yup I/O errors. we are going to have this disk replaced. it is part of vg02 (have 2 disks on this mirrored lvol)

# lvdisplay -v /dev/vg02/lvol1|more
--- Logical volumes ---
LV Name /dev/vg02/lvol1
VG Name /dev/vg02
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 7900
Current LE 1975
Allocated PE 3950
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c1t0d0 1975 1975
/dev/dsk/c2t0d0 1975 1975

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c1t0d0 00000 stale /dev/dsk/c2t0d0 00000 current
00001 /dev/dsk/c1t0d0 00001 stale /dev/dsk/c2t0d0 00001 current
00002 /dev/dsk/c1t0d0 00002 current /dev/dsk/c2t0d0 00002 current
00003 /dev/dsk/c1t0d0 00003 current /dev/dsk/c2t0d0 00003 current


so, can i do these steps...

shutdown the machine and replace drive(i know it is hot pluggable but we do this)
vgcfgrestore -n /dev/vg02 /dev/rdsk/c1t0d0
vgchange -a y vg02
vgsync vg02
apps can be brought up

is this OK?

Thanks

brian

[A. Clay Stephenson]

I haven't shutdown a box to replace a failed disk in at least 8 years; that's for wimps.


I would modify your procedure thusly:

Do an lvdisplay of each LVOL in this VG and make sure that all the extents on the good disk are current.


Slide the failing disk out of its slot a few centimeters and allow it to spin down. Leave it sitting there for about 2 minutes or so. This will have the effect of transforming your failing disk into a failed disk. Now remove the failing disk and insert the new disk. Wait about 60 seconds or so to allow the disk to come ready.


vgcfgrestore -n /dev/vg02 /dev/rdsk/c1t0d0
vgchange -a y /dev/vg02
vgsync /dev/vg02

[brian_31]

clay:

fully agreed. as i mentioned before it is a protocol we follow.

are the steps correct?

Thanks

brian

[A. Clay Stephenson]

Yes but you should still confirm that all the extents in every LVOL are current on the good disk. Just skip over the part about sliding the disk out and letting it spin down. Substitute, shutdown and replace the disk.

I am amazed that a production system can be brought down for such an ordinary task as replacing a failed disk.