topic Re: ioscan in Operating System - HP-UX

ioscan

antonio difruscia — Wed, 20 Dec 2006 08:20:54 GMT

I'm running HP-UX 11.23 with all the updated patches on an rx2600. when I boot or run ioscan it takes a long time (8 minutes to complete an ioscan).. after the ioscan completes all devices are claimed, I even ran
dd to all the devices with no problems.I am connected to an EVA3000 with luns presented to this node, and there is no hardware failures. also in the SAN there are 2 other EVAs with no luns presented to the node. Is there a way to run ioscan in a verbose mode so I can see what the node getting hung on during the scan ?

thanks
TonyD

Re: ioscan

My_Server — Wed, 20 Dec 2006 08:27:07 GMT

You can run the last config by excute ioscan -fnkC disk .
it's take just 1 or 2 sc

Re: ioscan

Andrew Merritt_2 — Wed, 20 Dec 2006 08:32:37 GMT

You could run it under tusc, and see where it is taking the time, e.g:

tusc -rall -wall -l -f -p -a -E -v -T%T /usr/sbin/ioscan -o /tmp/tusc.out

Those options should be enough to let you see where the time is being taken, and which device it is happening for.

Andrew

Re: ioscan

antonio difruscia — Wed, 20 Dec 2006 08:55:19 GMT

running funC doesn't help me because the systems run reboot tests and the system hangs when it does a ioscan on bootup, or when you run ioscan when they system is up
so I need to see what hangs the ioscan. The system does not have tusc loaded on it, so i'll see if I can locate tusc and try running that, it would be nice if they had
a verbose flag for ioscan.

thanks
TonyD

Re: ioscan

Torsten. — Wed, 20 Dec 2006 08:59:42 GMT

Hi,

on a large SAN the process could take some time. If you don't use the other 2 EVAs, you should prevent the access by adjusting your zoning and make the zone as small as possible.
This could speed up the discover process.

Another point to look at are failures in transmitting the data. You should look at your switch logs and search for high error rates.

Re: ioscan

antonio difruscia — Wed, 20 Dec 2006 09:16:58 GMT

there are no zones my san setup is a
2 core switches (4/16 brocade) with 8 edge
switches (2/16 brocade) the only error being
reported on the switches is a arptnew error
which I think is related to the switch trying to do a reverse arp to a system that is on a network and the switch can not reach
that node.

thanks
TonyD

Re: ioscan

spex — Wed, 20 Dec 2006 09:30:22 GMT

Hi Tony,

You could partition the ioscan by scanning a particular device or subset of devices at a time. Then, whichever scan(s) take(s) an inordinate amount of time is/are most likely the culprit(s).

For example, scan one hardware path at a time:
# ioscan -fnH 10/16/12.0.0
# ioscan -fnH 10/16/12.1.0
# ioscan -fnH 10/16/12.2.0

Or scan one device class at a time:
# ioscan -fnC ctl
# ioscan -fnC ext_bus
# ioscan -fnC lan

For additional clues, observe system tables in Glance during the scans.

PCS

Re: ioscan

Andrew Merritt_2 — Wed, 20 Dec 2006 10:02:51 GMT

One place to get tusc is here:
http://hpux.connect.org.uk/hppd/hpux/Sysadmin/tusc-7.8/

Andrew

Re: ioscan

Michael Steele_2 — Wed, 20 Dec 2006 10:24:57 GMT

Dear Tony:

This is a familiar problem to me. I have a rx2600 attached to a SAN / EMC disk array and this periodically happens. And it is always related to the disk array or SAN.

Run a 'insf -e' and you should be able to id the culprit device. It will be a fairly long report so be able to capture a couple of hundred lines.

Re: ioscan

Andrew Young_2 — Wed, 20 Dec 2006 10:45:11 GMT

Hi Tony

Please check if you have patch PHKL_33361 installed. We have had similar problems with an rx2620 and rx4640 connecting to MSA-1500's, one a direct connect and the other through a SAN, but both with the same problem.

Removing that patch solve these problems.

Alas it will mean a reboot.

Regards

Andrew Y

Re: ioscan

Andrew Young_2 — Wed, 20 Dec 2006 10:51:40 GMT

You might also want to look at this thread:

https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1031711

Re: ioscan

Sp4admin — Wed, 20 Dec 2006 11:02:00 GMT

Hello,

some times if you have exteranl disk this can cause the ioscan to take some time. does. all the devices come back caime

Re: ioscan

Torsten. — Wed, 20 Dec 2006 11:13:24 GMT

Look at the thread mentioned by Andrew.

The solution says
"Problem solved by modifying zone set configuration"

Especially in a large SAN you should consider zoning.

Without zoning, almost every devices talks to every other device. There is a lot of communication (let's call it "noise") in your SAN!

Every zone should only include a server and his storage (if possible). This help to make the SAN more easy, robust, fast and safe.

Re: ioscan

antonio difruscia — Wed, 20 Dec 2006 11:16:43 GMT

running ioscan -fnH on each hardware path
ran fine:

# time ioscan -fnH 0/5/1/0.5.15.0.0.0.1
Class I H/W Path Driver S/W State H/W Type Description
===========================================================================
disk 84 0/5/1/0.5.15.0.0.0.1 sdisk CLAIMED DEVICE HP HS0

real 0m0.03s
user 0m0.00s
sys 0m0.03s

but if I run ioscan -fnC lan or disk it takes over 8 minutes.

TonyD

Re: ioscan

Torsten. — Wed, 20 Dec 2006 13:22:47 GMT

ioscan -fnH does in fact display the kernel information (like "-k"), unless the target is a LBA or SBA.

See "man ioscan" for more information.

This is fast by default, you cannot compare this with an "life" scan ;-)

Re: ioscan

antonio difruscia — Wed, 20 Dec 2006 13:31:29 GMT

using the tusc command I get: out put in
attachment so it is readable. does anyone know what:
ioctl(4, IOCONF_IO_SCAN, 0x7ffff540)
points to ?

thanks
TonyD

Re: ioscan

antonio difruscia — Wed, 20 Dec 2006 13:46:26 GMT

I do have patch PHKL_33361 installed:

Is it documented anywhere about the removal of this patch ?

thanks
TonyD

Re: ioscan

Michael Steele_2 — Wed, 20 Dec 2006 16:12:10 GMT

Dear Tony:

You say this only happens on lan and disk ioscans, are you running NFS?

Can you id any tape issues? Try ioscan -fnC tape.

Please attach your complete 'ioscan' and these:

sar -d 5 5
sar -uM 5 5
insf -e > file

Re: ioscan

Andrew Young_2 — Thu, 21 Dec 2006 01:19:34 GMT

Hi.

Details on patch PHKL_33361 appear here:

http://www5.itrc.hp.com/service/cki/patchDocDisplay.do?patchId=PHKL_33361

It seems that you need to ensure that at least PHKL_31507 is installed.

However PHKL_33361 IS required if you are running a Superdome connected to EMC storage. Note that this patch fixes a memory leak in ioscan so running ioscan too often without this patch could cause problems.

So the debate around this patch is... if you need to run ioscan regularly are you prepared to deal with the memory leak vs performance.

Regards

Andrew Y

Re: ioscan

antonio difruscia — Thu, 21 Dec 2006 07:19:27 GMT

Mike, there are no tapes. files are attached
I have also found that more than just this
system showing the same problem. its when
ever you run ioscan without reading from kernel or on a bootup it takes 8 minutes.

thanks
TonyD