Fibre Channel Sanity check

[Richard Pereira_1]

Hi,

need a sanity check on a disk/SAN issue we're having. yesterday we seem to have lost all disk drives going through the same path (all report as no_hw in ioscan)
I have reset the card and my SAN group have told me they reset the ports and verified the cabling. I still have no_hw in ioscan, BUT fcmsutil shows me the following.


Vendor ID is = 0x00103c
Device ID is = 0x001029
XL2 Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x00128c
Topology = PTTOPT_FABRIC
Link Speed = 2Gb
Local N_Port_id is = 0x0b3900
N_Port Node World Wide Name = 0x50060b0000259f7b
N_Port Port World Wide Name = 0x50060b0000259f7a
Driver state = ONLINE
Hardware Path is = 0/0/14/1/0
Number of Assisted IOs = 156951433
Number of Active Login Sessions = 1
Dino Present on Card = NO
Maximum Frame Size = 2048
Driver Version = @(#) PATCH_11.11: libtd.a : Dec 12 2003, 14:34:15, PHSS_28509


doesn't this show the card as communicating to the switch? Since it can detect a topology, doesnt it show a fabric login occuring? I have asked them to recheck the zonning but they seem dubious and want me to either reboot the box or replace the FC card.

Can anyone suggest anything else to try before i pull everything apart?

thanks in advance,

[Patrick Wallek]

Yes, it does appear that you are at least talking to the switch.

I have a machine with a couple of fibre cards that aren't connected to anything. Attached is the fcmsutil output from one of them:


Vendor ID is = 0x00103c
Device ID is = 0x001029
XL2 Chip Revision No is = 2.2
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x00128c
Previous Topology = UNINITIALIZED
Link Speed = UNINITIALIZED
Local N_Port_id is = 0x000000
Local Loop_id is = 126
N_Port Node World Wide Name = 0x50060b0000112b97
N_Port Port World Wide Name = 0x50060b0000112b96
Driver state = AWAITING_LINK_UP
Hardware Path is = 0/6/2/0
Number of Assisted IOs = 0
Number of Active Login Sessions = 0
Dino Present on Card = NO
Maximum Frame Size = 960
Driver Version = @(#) PATCH_11.11: libtd.a : Jun 28 2002, 11:08:35, PHSS_26799


If you notice the "Driver State" on mine, it shows "Awaiting_Link_Up". Your shows online, so you should be OK from the FC card to the switch.

I would have the zoning checked. I don't think your problem is the card, cable, or GBIC in the switch.

[Robert Bennett_3]

I don't know your environment, but it sounds like the SAN folks might have changed something on you, maybe a controller card (just a guess, as only one path is showng NO_HW amd you obviously haven't changed anything).

I just did a major SAN upgrade and recieved all kinds of NO_HW's. My solution wsa to perform an ioscan -fn followed by an insf -e on each system affected. This didn't clear up the NO_HW's (only a reboot will remove them), but it did rediscover the new device addresses and CLAIMED them. When you perform an ioscan again, you will notice CLAIMED and NO_HW rather than just NO_HW. Then you'll have to figure out a good way time to reboot the systems to remove the NO_HW's.

Our SAN environment:
HP 9000's
McData Director Switch's
EVA 5000's

I hope this will work for you.

[Patrick Wallek]

Just an FYI -- You actually can remove the paths that are showing NO_HW via the rmsf command.

# rmsf -H hardware_path

# man rmsf

for details.

[Richard Pereira_1]

i thought about that, but ioscan doesnt list any new hw paths without device files. ill give a go anyways.

[Robert Bennett_3]

A note of caution on rmsf -H.

I tried this one time a while back and it hung our SuperDome - it was a very long night.
The system hung on I/O and I couldn't even perform a bdf. So since that time I have judged it easier to live with the NO_HW's until I could fanagle(sp) the time for a reboot.

[Jeff Schussele]

Hi Richard,

The fcmsutil output indicates that you can login to the server-side switch.
What about the array-side switch GBIC as well as the GBIC on the array itself?
So also have your storage people check those ports as well as the fibre between them.
Failure in any of those will give you these exact same symptoms.

My 2 cents,
Jeff