Ioscan waiting for systm - how to clear?

[A.Cooper]

Have a serviceguard setup, based on 2 L1500-7x servers. Problem is, cannot perform an ioscan using anything but cached data. If we do one, it just sits there, cannot be killed. Viewing the process via glance shows it's waiting on SYSTM.

Bit more info, until recently (well, beginning of Jan) we were getting frequent SCSI lbolt reset notification from EMS, referring to different disks in the disc pack. HP engineers couldn't fix the problem, but they eventually stopped. Can't help thinking these may be the cause of our dodgy ioscan.

Also, o/s is hp-ux 11.11. Was thinking of installing PHKL_29047, which fixes ioscan hangs if bad disks are present, but we have no indictaion that any of our disks are bad!

How can I fix our ioscan problem?

Cheers,

Matt.

[RAC_1]

Run sam and through sam try to get details for all disks on system. If it fails SAMlog will tell you something.

Also to check any of your disk are bad run diskinfo -v /dev/rdsk/cxtxdx on all disks.

Check if it gives any error.

But installing the patch is good idea. It is always good to be up to date on patches.

[A.Cooper]

The ioscan that sam was trying to do is hanging now!

And sam is hanging because of it!

diskinfo -v doesn't give any errors.

Cheers,

Matt.

[Elmar P. Kolkman]

From your description I would conclude that the EMS messages and hanging ioscan are good indications of broken disks or SCSI controller (don't forget those).

Also, look at dmesg for possible interesting messages.

Does the ioscan from cache show all disks as claimed, or are some of them shown as NO_HW?
And, if you mirror using LVM, have you checked for stale blocks?

[Suresh Patoria]

Hi,

Check the dmesg log

it may give u errors generated by disk

Thanx

[A.Cooper]

dmesg is attached. This says PV1 returned to VG 1, which is /dev/dsk/c4t0d0. The cached ioscan shows this as CLAIMED. pvdisplay -v sucessful, vgdisplay of a vg on the pv shows no errors. lvdisplay -v also shows no stale extents.

I think this dmesg is from back in Jan, the syslog has similar entries around that time (we haven't had any further resets since 5th Jan).

When we first got these scsi resets we increased pvtimeout to 180 secs. Didn't work. HP engineer replaced entire disk enclosure and all disks. Still happened :-/

Thanks for all the ideas so far....

Matt.

[A.Cooper]

Just spotted in ioscan -fu, that one disc enclosure is in state SCAN.

Reckon that's it?

ctl 8 0/8/0/1.15.0 sctl SCAN DEVICE HP A6491A

[Bill Hassell]

ioscan (without -k) requires that SCSI ID commands and queries complete properly and there are no timeouts. So you definitely have issues with the hardware. You can verify that other items are OK by limiting the full scan to just lan or tape:

ioscan -fC lan
ioscan -fC tape

Then try limiting ioscan to just a hardware path:

ioscan -fH 10/8.9.0

or similar. You might have to start with just the This may get it down to the top part of the path and work down:

ioscan -fH 10
ioscan -fH 10/8

and so on. Note that a bad disk is in the eye of the beholder. It may not produce parity errors or seek failures but the electronics that reports back extended status may be broken, yet data gets transferred OK. And of course, there may be some firmware issue(s) in your disk arrays.

[A.Cooper]

Just to let you know how this was resolved.
The ioscan was hanging because of SCSI resets.
We were getting SCSI resets, because HP had installed ILT cables for use with our DS2300's and A5150A SCSI cards.
These cables should not be used in Serviceguard type setups. Changed the cables to normal non terminating ones, and all sorted!