ct_query failed errors in syslog

[Venkat_11]

Hi All,

We have a HP rp 3440. And we assigned storage from Hitachi through HP Tachyon XL2 Fibre Channel Mass Storage Adapter. And recently we reclaimed the storage which is assigned to the server and after that are getting error messages to the syslog file like this.


May 15 05:05:00 usxsl004 vmunix: ct_query failed. hw_path = 0/3/1/0
May 15 05:05:00 usxsl004 vmunix: ct_query failed. hw_path = 0/4/1/0

usxsl004:root# ioscan -funC fc
Class I H/W Path Driver S/W State H/W Type Description
=================================================================
fc 0 0/3/1/0 td CLAIMED INTERFACE HP Tachyon XL2 Fibre Channel Mass Storage Adapter
/dev/td0
fc 1 0/4/1/0 td CLAIMED INTERFACE HP Tachyon XL2 Fibre Channel Mass Storage Adapter
/dev/td1

And when i ran the follwoing command i found the NO_HW

usxsl004:root# ioscan -fnH 0/3/1/0
Class I H/W Path Driver S/W State H/W Type Description
==========================================================================
fc 0 0/3/1/0 td CLAIMED INTERFACE HP Tachyon XL2 Fibre Channel Mass Storage Adapter
/dev/td0
fcp 0 0/3/1/0.200 fcp NO_HW INTERFACE FCP Domain
ext_bus 4 0/3/1/0.200.0.1.0 fcparray NO_HW INTERFACE FCP Array Interface
target 7 0/3/1/0.200.0.1.0.0 tgt NO_HW DEVICE
target 8 0/3/1/0.200.0.1.0.1 tgt NO_HW DEVICE
ext_bus 5 0/3/1/0.200.0.255.0 fcpdev NO_HW INTERFACE FCP Device Interface
target 9 0/3/1/0.200.0.255.0.1 tgt NO_HW DEVICE


usxsl004:root# ioscan -fnH 0/4/1/0
Class I H/W Path Driver S/W State H/W Type Description
==========================================================================
fc 1 0/4/1/0 td CLAIMED INTERFACE HP Tachyon XL2 Fibre Channel Mass Storage Adapter
/dev/td1
fcp 1 0/4/1/0.179 fcp NO_HW INTERFACE FCP Domain
ext_bus 6 0/4/1/0.179.2.10.0 fcparray NO_HW INTERFACE FCP Array Interface
target 10 0/4/1/0.179.2.10.0.0 tgt NO_HW DEVICE
target 11 0/4/1/0.179.2.10.0.1 tgt NO_HW DEVICE
ext_bus 7 0/4/1/0.179.2.255.0 fcpdev NO_HW INTERFACE FCP Device Interface
target 12 0/4/1/0.179.2.255.0.10 tgt NO_HW DEVICE

Can anybody tell how to get rid o f these error messages.

Please let me know if you need any more information.

Thanks a lot

Regards
Venkat

[Steven E. Protter]

Shalom,

A disk has failed or been taken away by the SAN admin.

dmesg -

wait

dmesg

And see if it stays away. You may need to boot.

SEP

[Venkat_11]

Is rebooting is the only solution to get rid of this errors? ...is there any other way to make this online with out bringing down the server.

d device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
0/4/1/0: Unable to access previously accessed device at nport ID 0xb3020a.
0/3/1/0: Unable to access previously accessed device at nport ID 0xc80001.
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0
ct_query failed. hw_path = 0/3/1/0
ct_query failed. hw_path = 0/4/1/0

Thanks
Venkat

[Sameer_Nirmal]

Hi Venkat,

The "ct_query failed" messages indicates no devices seen/found through the FC HBAs.
The same is reflected as "NO_HW" in ioscan output.

Are those HBAs connected to a switch? or just not-connected?

If they are connected to a switch ,you need to check if devices are configured. The device could be storage lun etc. Check if these HBAs are part of a zone ( fabric zone) and a device is configured in the same zone.

[Venkat_11]

Hi,

The HBA's are connecetd to the fiber switch but on the switch the ports are disabled, because these HBA's are not presenting any luns to the server. Earlier they use to present but right now they are recliamed. Before disabling the ports on the Fabric switch side the Zonning was cleared.

Please let me know if you need any more information.

Thanks
Venkat

[Sameer_Nirmal]

Venkat,

The picture is clear now...

As you said, the HBAs are not in use. The messages are being display by the kernel through FC driver, as those entries for previously configured devices exists in the name servers on the cards.

You can try following.

# fcmsutil /dev/td[x] clear_devstat all
# fcmsutil /dev/td[x] clear_nsstat
# fcmsutil /dev/td[x] -f disable

[Florian Heigl (new acc)]

You *might* be successful by removing all outdated device file entries in /dev/*dsk/

(You can find them using 'lssf $devfile | grep "???" ' )

and after that issue ioscan -fnCdisk

The kernel *should* now drop the devices from its memory.

[Venkat_11]

Hi,

I didnt see any entries in /dev/dsk apart from the local disks

usxsl004:root# ls -l /dev/dsk/
total 0
brw-r----- 1 bin sys 31 0x000000 Sep 29 2005 c0t0d0
brw-r----- 1 bin sys 31 0x020000 Sep 29 2005 c2t0d0
brw-r----- 1 bin sys 31 0x021000 Sep 29 2005 c2t1d0

Any better ideas to get rid of the error messages.

Thanks and Regards
Venkat