fc errors messages

[Dewa Negara_4]

Hi All,

I got the following error messages related to fc card from dmesg command and syslog.log. Anyone can help me what actually the issue and how to resolve it? will rebooting the server solve the issue? This server hasn't been rebooted since 267 days. Also I found some tape devices are on NO_HW status.

Please help. Thanks for your help in advance.

Thanks and Best Regards,
Negara

0/5/0/0: Unable to access previously accessed device at nport ID 0x270d26.
0/5/0/0: 'World-wide name' (unique identifier) for device at nport ID 0x270eef
changed. If the device has been replaced intentionally, please use the
fcmsutil(1M) command's replace_dsk option to allow the new device to be used.
0/5/0/0: Fibre Channel Driver detected a parse error in the FLOGI/PLOGI response
returned by nport ID 0x270eef. FLOGI/PLOGI Fail Code = 0x6.

[Steven E. Protter]

With regards to the tape devices.

They have most likely failed.

It would appear that one of the fc devices has been replaced. Pretty hard to figure out how if the server has not been downed for nearly a year.

But thats what is being indicated.

When you replace such a card, the World Wide Name, which is like a mac address has to be adjsuted.

We usually do it in the san configuration.

Since the tape turned up bad at the same time as the fc problem I will surmise that some part of your bus has failed and the address scheme has been messed up.

I'd run dmesg, mstm,cstm or xstm and try and find the point of failure. Then I'd get the server serviced.

Good Luck,

SEP

[steven Burgess_2]

Hello

You have lost some hardware in this case possibly a disk or controller

please paste output of

ioscan -fnH 0/5/0/0

If this devices shows as NO_HW then you may have lost one side of the path to your fibre disks. If this is claimed then

ioscan -fnC disk

and attach output

Regards

Steve

[Dewa Negara_4]

Hi Steven,

Thanks for your information. Please find the attached output below. Thanks for your help.

Best Regards,
Negara

Class I H/W Path Driver S/W State H/W Type Descrn
================================================================================
fc 0 0/5/0/0 td CLAIMED INTERFACE HP Tar
/dev/td0
fcp 0 0/5/0/0.8 fcp NO_HW INTERFACE FCP Pr
ext_bus 7 0/5/0/0.8.0.1.0 fcparray NO_HW INTERFACE FCP Ae
target 6 0/5/0/0.8.0.1.0.10 tgt NO_HW DEVICE
ext_bus 8 0/5/0/0.8.0.1.1 fcparray NO_HW INTERFACE FCP Ae
target 7 0/5/0/0.8.0.1.1.11 tgt NO_HW DEVICE
ext_bus 9 0/5/0/0.8.0.255.0 fcpdev NO_HW INTERFACE FCP De
target 8 0/5/0/0.8.0.255.0.1 tgt NO_HW DEVICE
ctl 7 0/5/0/0.8.0.255.0.1.0 sctl NO_HW DEVICE CROSS0
/dev/rscsi/c9t1d0
fcp 1 0/5/0/0.39 fcp CLAIMED INTERFACE FCP Dn
ext_bus 11 0/5/0/0.39.0.255.2 fcpdev NO_HW INTERFACE FCP De
target 34 0/5/0/0.39.0.255.2.6 tgt NO_HW DEVICE
tape 2 0/5/0/0.39.0.255.2.6.0 stape NO_HW DEVICE IBM 1
/dev/rmt/2m
/dev/rmt/2mb
/dev/rmt/2mn
/dev/rmt/2mnb
/dev/rmt/c11t6d0BEST
/dev/rmt/c11t6d0BESTb
/dev/rmt/c11t6d0BESTn
/dev/rmt/c11t6d0BESTnb
/dev/rmt/c11t6d0cnb
ext_bus 33 0/5/0/0.39.0.255.14 fcpdev CLAIMED INTERFACE FCP De
target 56 0/5/0/0.39.0.255.14.15 tgt CLAIMED DEVICE
tape 1 0/5/0/0.39.0.255.14.15.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/1m
/dev/rmt/1mb
/dev/rmt/1mn
/dev/rmt/1mnb
/dev/rmt/c33t15d0BEST
/dev/rmt/c33t15d0BESTb
/dev/rmt/c33t15d0BESTn
/dev/rmt/c33t15d0BESTnb
ext_bus 12 0/5/0/0.39.1.255.2 fcpdev CLAIMED INTERFACE FCP De
target 35 0/5/0/0.39.1.255.2.6 tgt CLAIMED DEVICE
tape 3 0/5/0/0.39.1.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/3m
/dev/rmt/3mb
/dev/rmt/3mn
/dev/rmt/3mnb
/dev/rmt/c12t6d0BEST
/dev/rmt/c12t6d0BESTb
/dev/rmt/c12t6d0BESTn
/dev/rmt/c12t6d0BESTnb
/dev/rmt/c12t6d0cnb
ext_bus 13 0/5/0/0.39.2.255.2 fcpdev NO_HW INTERFACE FCP De
target 36 0/5/0/0.39.2.255.2.6 tgt NO_HW DEVICE
tape 4 0/5/0/0.39.2.255.2.6.0 stape NO_HW DEVICE IBM 1
/dev/rmt/4m
/dev/rmt/4mb
/dev/rmt/4mn
/dev/rmt/4mnb
/dev/rmt/c13t6d0BEST
/dev/rmt/c13t6d0BESTb
/dev/rmt/c13t6d0BESTn
/dev/rmt/c13t6d0BESTnb
/dev/rmt/c13t6d0cnb
ext_bus 34 0/5/0/0.39.2.255.14 fcpdev CLAIMED INTERFACE FCP De
target 57 0/5/0/0.39.2.255.14.15 tgt CLAIMED DEVICE
tape 8 0/5/0/0.39.2.255.14.15.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/8m
/dev/rmt/8mb
/dev/rmt/8mn
/dev/rmt/8mnb
/dev/rmt/c34t15d0BEST
/dev/rmt/c34t15d0BESTb
/dev/rmt/c34t15d0BESTn
/dev/rmt/c34t15d0BESTnb
ext_bus 14 0/5/0/0.39.3.255.2 fcpdev CLAIMED INTERFACE FCP De
target 37 0/5/0/0.39.3.255.2.6 tgt CLAIMED DEVICE
tape 5 0/5/0/0.39.3.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/5m
/dev/rmt/5mb
/dev/rmt/5mn
/dev/rmt/5mnb
/dev/rmt/c14t6d0BEST
/dev/rmt/c14t6d0BESTb
/dev/rmt/c14t6d0BESTn
/dev/rmt/c14t6d0BESTnb
/dev/rmt/c14t6d0cnb
ext_bus 15 0/5/0/0.39.4.255.13 fcpdev CLAIMED INTERFACE FCP De
target 38 0/5/0/0.39.4.255.13.5 tgt CLAIMED DEVICE
tape 6 0/5/0/0.39.4.255.13.5.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/6m
/dev/rmt/6mb
/dev/rmt/6mn
/dev/rmt/6mnb
/dev/rmt/c15t5d0BEST
/dev/rmt/c15t5d0BESTb
/dev/rmt/c15t5d0BESTn
/dev/rmt/c15t5d0BESTnb
/dev/rmt/c15t5d0cnb
ext_bus 16 0/5/0/0.39.5.255.2 fcpdev CLAIMED INTERFACE FCP De
target 39 0/5/0/0.39.5.255.2.6 tgt CLAIMED DEVICE
tape 7 0/5/0/0.39.5.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/7m
/dev/rmt/7mb
/dev/rmt/7mn
/dev/rmt/7mnb
/dev/rmt/c16t6d0BEST
/dev/rmt/c16t6d0BESTb
/dev/rmt/c16t6d0BESTn
/dev/rmt/c16t6d0BESTnb
/dev/rmt/c16t6d0cnb
ext_bus 17 0/5/0/0.39.6.255.2 fcpdev NO_HW INTERFACE FCP De
target 40 0/5/0/0.39.6.255.2.6 tgt NO_HW DEVICE
ext_bus 31 0/5/0/0.39.6.255.14 fcpdev CLAIMED INTERFACE FCP De
target 54 0/5/0/0.39.6.255.14.15 tgt CLAIMED DEVICE
tape 0 0/5/0/0.39.6.255.14.15.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/0m
/dev/rmt/0mb
/dev/rmt/0mn
/dev/rmt/0mnb
/dev/rmt/c31t15d0BEST
/dev/rmt/c31t15d0BESTb
/dev/rmt/c31t15d0BESTn
/dev/rmt/c31t15d0BESTnb
/dev/rmt/c31t15d0cnb
ext_bus 18 0/5/0/0.39.7.255.2 fcpdev CLAIMED INTERFACE FCP De
target 41 0/5/0/0.39.7.255.2.6 tgt CLAIMED DEVICE
tape 9 0/5/0/0.39.7.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/9m
/dev/rmt/9mb
/dev/rmt/9mn
/dev/rmt/9mnb
/dev/rmt/c18t6d0BEST
/dev/rmt/c18t6d0BESTb
/dev/rmt/c18t6d0BESTn
/dev/rmt/c18t6d0BESTnb
/dev/rmt/c18t6d0cnb
ext_bus 19 0/5/0/0.39.8.255.2 fcpdev CLAIMED INTERFACE FCP De
target 42 0/5/0/0.39.8.255.2.6 tgt CLAIMED DEVICE
tape 10 0/5/0/0.39.8.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/10m
/dev/rmt/10mb
/dev/rmt/10mn
/dev/rmt/10mnb
/dev/rmt/c19t6d0BEST
/dev/rmt/c19t6d0BESTb
/dev/rmt/c19t6d0BESTn
/dev/rmt/c19t6d0BESTnb
/dev/rmt/c19t6d0cnb
ext_bus 20 0/5/0/0.39.9.255.2 fcpdev CLAIMED INTERFACE FCP De
target 43 0/5/0/0.39.9.255.2.6 tgt CLAIMED DEVICE
tape 11 0/5/0/0.39.9.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/11m
/dev/rmt/11mb
/dev/rmt/11mn
/dev/rmt/11mnb
/dev/rmt/c20t6d0BEST
/dev/rmt/c20t6d0BESTb
/dev/rmt/c20t6d0BESTn
/dev/rmt/c20t6d0BESTnb
/dev/rmt/c20t6d0cnb
ext_bus 21 0/5/0/0.39.10.255.2 fcpdev CLAIMED INTERFACE FCP De
target 44 0/5/0/0.39.10.255.2.6 tgt CLAIMED DEVICE
tape 12 0/5/0/0.39.10.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/12m
/dev/rmt/12mb
/dev/rmt/12mn
/dev/rmt/12mnb
/dev/rmt/c21t6d0BEST
/dev/rmt/c21t6d0BESTb
/dev/rmt/c21t6d0BESTn
/dev/rmt/c21t6d0BESTnb
/dev/rmt/c21t6d0cnb
ext_bus 22 0/5/0/0.39.11.255.2 fcpdev CLAIMED INTERFACE FCP De
target 45 0/5/0/0.39.11.255.2.6 tgt CLAIMED DEVICE
tape 13 0/5/0/0.39.11.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/13m
/dev/rmt/13mb
/dev/rmt/13mn
/dev/rmt/13mnb
/dev/rmt/c22t6d0BEST
/dev/rmt/c22t6d0BESTb
/dev/rmt/c22t6d0BESTn
/dev/rmt/c22t6d0BESTnb
/dev/rmt/c22t6d0cnb
ext_bus 23 0/5/0/0.39.12.255.2 fcpdev CLAIMED INTERFACE FCP De
target 46 0/5/0/0.39.12.255.2.6 tgt CLAIMED DEVICE
tape 14 0/5/0/0.39.12.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/14m
/dev/rmt/14mb
/dev/rmt/14mn
/dev/rmt/14mnb
/dev/rmt/c23t6d0BEST
/dev/rmt/c23t6d0BESTb
/dev/rmt/c23t6d0BESTn
/dev/rmt/c23t6d0BESTnb
/dev/rmt/c23t6d0cnb
ext_bus 24 0/5/0/0.39.13.255.2 fcpdev NO_HW INTERFACE FCP De
target 47 0/5/0/0.39.13.255.2.6 tgt NO_HW DEVICE
tape 15 0/5/0/0.39.13.255.2.6.0 stape NO_HW DEVICE IBM 1
/dev/rmt/15m
/dev/rmt/15mb
/dev/rmt/15mn
/dev/rmt/15mnb
/dev/rmt/c24t6d0BEST
/dev/rmt/c24t6d0BESTb
/dev/rmt/c24t6d0BESTn
/dev/rmt/c24t6d0BESTnb
/dev/rmt/c24t6d0cnb
ext_bus 25 0/5/0/0.39.14.255.2 fcpdev NO_HW INTERFACE FCP De
target 48 0/5/0/0.39.14.255.2.6 tgt NO_HW DEVICE
tape 16 0/5/0/0.39.14.255.2.6.0 stape NO_HW DEVICE IBM 1
/dev/rmt/16m
/dev/rmt/16mb
/dev/rmt/16mn
/dev/rmt/16mnb
/dev/rmt/c25t6d0BEST
/dev/rmt/c25t6d0BESTb
/dev/rmt/c25t6d0BESTn
/dev/rmt/c25t6d0BESTnb
/dev/rmt/c25t6d0cnb
ext_bus 32 0/5/0/0.39.14.255.14 fcpdev NO_HW INTERFACE FCP De
target 55 0/5/0/0.39.14.255.14.15 tgt NO_HW DEVICE
ext_bus 26 0/5/0/0.39.15.255.2 fcpdev CLAIMED INTERFACE FCP De
target 49 0/5/0/0.39.15.255.2.6 tgt CLAIMED DEVICE
tape 17 0/5/0/0.39.15.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/17m
/dev/rmt/17mb
/dev/rmt/17mn
/dev/rmt/17mnb
/dev/rmt/c26t6d0BEST
/dev/rmt/c26t6d0BESTb
/dev/rmt/c26t6d0BESTn
/dev/rmt/c26t6d0BESTnb
/dev/rmt/c26t6d0cnb
ext_bus 27 0/5/0/0.39.16.255.2 fcpdev CLAIMED INTERFACE FCP De
target 50 0/5/0/0.39.16.255.2.6 tgt CLAIMED DEVICE
tape 18 0/5/0/0.39.16.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/18m
/dev/rmt/18mb
/dev/rmt/18mn
/dev/rmt/18mnb
/dev/rmt/c27t6d0BEST
/dev/rmt/c27t6d0BESTb
/dev/rmt/c27t6d0BESTn
/dev/rmt/c27t6d0BESTnb
/dev/rmt/c27t6d0cnb
ext_bus 28 0/5/0/0.39.17.255.2 fcpdev CLAIMED INTERFACE FCP De
target 51 0/5/0/0.39.17.255.2.6 tgt CLAIMED DEVICE
tape 19 0/5/0/0.39.17.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/19m
/dev/rmt/19mb
/dev/rmt/19mn
/dev/rmt/19mnb
/dev/rmt/c28t6d0BEST
/dev/rmt/c28t6d0BESTb
/dev/rmt/c28t6d0BESTn
/dev/rmt/c28t6d0BESTnb
/dev/rmt/c28t6d0cnb
ext_bus 29 0/5/0/0.39.18.255.2 fcpdev NO_HW INTERFACE FCP De
target 52 0/5/0/0.39.18.255.2.6 tgt NO_HW DEVICE
tape 20 0/5/0/0.39.18.255.2.6.0 stape NO_HW DEVICE IBM 1
/dev/rmt/20m
/dev/rmt/20mb
/dev/rmt/20mn
/dev/rmt/20mnb
/dev/rmt/c29t6d0BEST
/dev/rmt/c29t6d0BESTb
/dev/rmt/c29t6d0BESTn
/dev/rmt/c29t6d0BESTnb
/dev/rmt/c29t6d0cnb
ext_bus 35 0/5/0/0.39.18.255.14 fcpdev CLAIMED INTERFACE FCP De
target 58 0/5/0/0.39.18.255.14.15 tgt CLAIMED DEVICE
tape 22 0/5/0/0.39.18.255.14.15.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/22m
/dev/rmt/22mb
/dev/rmt/22mn
/dev/rmt/22mnb
/dev/rmt/c35t15d0BEST
/dev/rmt/c35t15d0BESTb
/dev/rmt/c35t15d0BESTn
/dev/rmt/c35t15d0BESTnb
ext_bus 30 0/5/0/0.39.19.255.2 fcpdev CLAIMED INTERFACE FCP De
target 53 0/5/0/0.39.19.255.2.6 tgt CLAIMED DEVICE
tape 21 0/5/0/0.39.19.255.2.6.0 stape CLAIMED DEVICE IBM 1
/dev/rmt/21m
/dev/rmt/21mb
/dev/rmt/21mn
/dev/rmt/21mnb
/dev/rmt/c30t6d0BEST
/dev/rmt/c30t6d0BESTb
/dev/rmt/c30t6d0BESTn
/dev/rmt/c30t6d0BESTnb
/dev/rmt/c30t6d0cnb

[steven Burgess_2]

Hi

Sorry for the lat response, have been extremely busy

Have you got an external tape robot attached to the server. It looks to me like it may be having problems with one if it's controllers. We have an adic here which has similar types of issues every now and then where we lose one the paths to the device. Once we reset the SNC it cures the problem

If you look at your ioscan and sort by the hardware address things look a little clearer

If it's not the robot, it's the fcp on

0/5/0/0.8

Going home now so won't be able to give any more input. Hope you get it sorted

Rgds

Steve

HTH

Steve

[Gabriele FACCHINI · Accepted solution]

Hi if you want to make that the massage don't appear in the syslog you must to launch this command:
fcmsutil /dev/td0 replace_dsk 0x270d26

by
Gabriele

[Dewa Negara_4]

Hi Gabriele,

Thanks for your help. Could you please help to elaborate further? What actually happen with my system? Why the nport ID was changed? and how about some hardware with NO_HW status? Will they be able to resolve as well after executing your command? Please help.

Thanks and Best Regards,
Negara

[Gabriele FACCHINI]

Hi Dewa, if you launch the command that I write you one device return in the state CLAIMED. After you must to launch this command :
ioscan -fn
more /var/adm/sysslog/syslog.log
and you must to see in the last lines if compare another line like "0/5/0/0: 'World-wide name' (unique identifier) for device at nport ID 0x270eef". If compare you must to launch:
fcmsutil /dev/td0 relace_dsk 0x270eef
after you can relaunch:
ioscan -fn
and see if all the tape device are in the CLAIMED state.
I hope to help you.
by
Gabriele

[Steven E. Protter]

The tape robot might not be claimed if there is no tape in it. Not a lot of experience there.

We print off the ioscan commands into our DR box. That gives me something to compare with(because i always keep a copy in the print queue)when trouble strikes.

A good idea from next time.

Reading your output says to me controller failure.

SE{

[Gabriele FACCHINI]

Hi Dewa, probably the devices connected to the ports 13 and 14 was replaced, and then the wwn of the device was changed and the server can not contact them. If you launch the command fcmsutil with the option replace_dsk you specify that no authentication should be performed the next time we communicate with the device, and the system can find this device.
If you need more help write me.
by
Gabriele