ioscan takes too long to display output on HPux 11.23 PA box

[Natesan_1]

Hardware configuration:
-----------------------
RP5430-2X750Mhz-->PDC ver:44.12, GSP ver:B2.20
2GB FCA card A6795A --> driver & FC cumulative patched as on mar-06
SAN switch-->Kernel:5.4,Fabric OS: v3.1.3a
EVA5000 --> VCS 3028

Software configuration:
----------------------
HPUx 11.23 --> OS mar-06 OS media
Patch level--> Mar-06 QPKbase +apps +Feature sets.
Securepath 3.0F


Problem:
--------
ioscan takes too long time to return the output.

Action taken:
------------
1.Checked compatiable PDC, SAN Secure path versions found to be Ok.
2.This problem was there before installing the securepath 3.0F also
3.Disconnecting both FC ports, ioscan works fine.(problem norrowdown to FC)
4.We have another server L2000 (with PDC ver:44.28 with similar FC card & OS 11.23, Mar-06 patch level)ioscan works fine. (Surprise !!!)

Action pending:
---------------
1. Planning to install latest FC driver& cummulative patches, ioscan related patches
which found to be missing.(Note: those patches release note doesn't talk about problem /fix which i am facing.)

Help:
-------
If some one has similar problem & had solution/ suggestions/workaround please help me by sharing.

Thanks in advance for your reply.

[Natesan_1]

No Errors reported,
syslog, stm-logtool out, dmesg etc,
all patches and softwares are in configured state.
even after taking ioscan too long to return o/p, all hw/sw states are CLIMED ...

[Torsten.]

It takes a while, if the SAN is large. How long it takes?

Perform your pending actions and give another try.

[James R. Ferguson]

Hi:

This may go without saying, but unless you really need to rescan the hardware, do your 'ioscan' using the kernel's current tables:

# ioscan -kfn

(as opposed to)

# ioscan -fn

Regards!

...JRF...

[Joseph C. Denman]

James is correct, as he usually is. ioscan can vary depending on the hardware components and system type. It can take quite a while. It is much faster to grab the information from the kernal as stated above.

...jcd...

[Natesan_1]

Hi

Thanks for your reply. it takes almost 3 to 5 minutes to give reply. whereas all the other servers in SAN gives immediate reply.
As I said already that another PA-RISC server L2000 also have the similar OS +patching give ioscan output immeditate.

in this server some things strange ... !!

[Sheriff Andy]

When you run ioscan, does it make a big impact on performance (view thru top)?

Also, how many HBA cards do you have attached to this EVA?

[Bill Hassell]

As mentioned, ioscan -k will give you instantaneous results every time and unless you have installed new hardware with power on, there is no reason not to use -k. Delays occur in ioscan due to driver discovery steps. You can isolate the scan delays by specifying -H to a specific path so generally speaking, driver patches (and occasionally ioscan) PHCO_31603 is the latest for ioscan.

[sathish kannan]

Hello Natesan,

I have seen instance where "ioscan" can take longer time when you have LUNs in buisness copy position on SAN environmnets. Are all the disk in Ready mode/read write mode?

Check the status of the LUNS presented to the host?

Regards
Sathish

[Natesan_1]

Hi Folks,

Thanks for your reply,

ioscan -k option is not the solution please stop recommending the same.Since similar server connected to SAN having no delay in ioscan.There is a problem let us try to solve it.

1.We have no luns / BC luns presented from EVA to this server.
2.Top & glance there is no remarkable CPU/MEM/IO load rise when applying ioscan.
3.After too much delay(ioscan) output shows about EVA controllers and all I/O etc without any discrepency.

[Sandman!]

Can you post the output of "ioscan -fnC disk" as it might help in debugging.

[Darrel Louis]

Natesan,

This could be a software or hardware problem.
As you already stated to install patches etc, please check for the following patches:
PHKL_33361 (Cumulative kernel SCSI patch)
Performance degradation.

Do you see the problem described in patch PHKL_34479 (WSIO (IO) subsystem MSI/MSI-X/WC Patch)
( SR:8606428507 CR:JAGaf87988 )
Running ioscan(1M) repeatedly, without any hardware configuration changes, on a system with attached FibreChannel devices causes memory leaks.

Just for our understanding and feeling, what's the time difference between:
"ioscan -fnkCDisk " and "ioscan -fnCdisk"

Darrel

[Natesan_1]

Hi Darrel,

Difference is very high about 3-5minutes compared to kernel mode(-k).Also if I disconnect the both the FCA's. ioscan -fn and ioscan -kfn no difference, working fine.

I will update patch and let u know the result.

[Steven E. Protter]

Shalom,

I'd suggest searching 11.23 patch archive for ioscan. Then read the patch notes and pick one that fits your issue along with dependencies.

SEP

[Darrel Louis]

Natesan,

I also suggest to run a patch assesment, to check which patches on your system have a critical alert.

Darrel

[Mridul Shrivastava]

Run Patch Assessment


1) open the page ==> http://www1.itrc.hp.com/service/patch/assessSystemsPage.do?BC=main|
2) click on "select a system to assess or upload new system information"
3) Download the script "swainv ". Right click on this and save to the system
4) Transfer this file to the HP-UX system. If ftp is used, then select binary mode of transfer.
5) Go to the directory in HP-UX system where the script "swainv " is loaded and give the executable permission by using the command " chmod 700 swainv "
6) Execute the script as " ./swainv "
7) This will generate a file called "inventory.xml" in current directory.
8) Transfer the file to the system which has got access to ITRC site.
9) Use the above link " http://www1.itrc.hp.com/service/patch/assessSystemsPage.do?BC=main " again and upload the "inventory.xml" to ITRC by clicking "submit".
10) click on " display candidate patches". The site will give the recommendations.
11) click on "create assessment report". This will give consolidated report.
12) Check at the line " Patches recommended by InstalledPatchAnalyzer (summary)" for recommendations.

[Michael Steele_2]

Really think you've got some kind of HW issue going on and that you should check you HW logs for errors. Critical EMS entries will be in syslog.log or OLDsyslog.log. Can you paste any of these in? More dependable are errors in LOGTOOL and I'd definitely like to see these pasted in.

STM > TOOLS > UTILITY > RUN > LOGTOOL > FILE > VIEW > RAW SUMMARY.

Hope this helps mon ami.

[Sandman!]

Hi Natesan,

Could be a problem with your SAN fabric. Is the good L2000 server also going through the same fabric? How many LUNs can you see on the EVA5000?

It would really help if you posted the following output here:

# ioscan -fnC disk

[Sandman!]

The following reports are going to help too...

# ioscan -fnC fc
# ioscan -fnC fcp

thanks!

[support_5]

We have a similar problem where ioscan takes three minutes to return. It may be a patch thing where a patch may rectify the issue. It appears to be some sort of time-out within the ioscan command...perhaps waiting for some removed hardware to return, eg disk presented from disk-array, or perhaps securepath or autopath or something is playing up. We're still looking into the problem here, so no answers unfortunately.

Ta

[Natesan_1]

Hi All,

Thanks for your kind co-operation and timely reply.

I have installed latest patches of Fibre channel and SCSI cumulative But no luck.


Problem solved by modifying zone set configuration.

old zone config:
===============
All hpux server ports and EVA ports into HPUXzone_disk zone set
All hpux server ports and MSL-NSR port info HPUX backup_tape zone set.

Modification config
===================
removed the problematic system from above zone set and created new two zones set

problemtic serverport and EVA port into zone set1
problemtic serverport and MSL-NSR port into
zone set2

now ioscan works fine.

unfortunatly we didn't suspect this at first time due to all other servers were working fine.(mix of PA and IA 11.23 server on same zone sets)

Anyway thanks for all who share their experiences on this problem.

[Natesan_1]

Creating Separate zone sets for server with disk and server with tape solved the problem.