BladeSystem - General
1747988 Members
4816 Online
108756 Solutions
New Discussion юеВ

C7000 FC-VC Modules dropping

 
mark_uksouth
Frequent Advisor

C7000 FC-VC Modules dropping

Hi all,
We are having an intermittant problem with one of our C7000's.
OA firmware 2.21
VC-Enet firmware 1.30
VC-FC firmware 1.20
Once a week or so the VC-FC's will drop connections on the HBA's which we then have to force them to re logon.
This seems to only be happening on the server profiles that are connected to uplink 1 & 2.
We have VDI esx boxes on Uplink 4 which have no issues.
All uplinks show as being logged in.
Also I noticed this morning when after it happened yesterday that esx server profiles connected to uplink 2 were missing what uplink they connect to in the server port information section of VC.
But strangly I could still browse some or the datastores through esx and some virtual machine were still running?.
I have tried port speeds as either 4gb or auto but this doesnt make any difference.

Looking at the VC syslog it shows the following:

2008-08-31T18:27:34+00:00 [ENC:Unknown:2013:Critical] Enclosure state FAILED : Component is not operational due to an error
2008-08-31T18:27:34+00:00 [VCD:xxxx:1022:Critical] Domain state FAILED : Component is not operational due to an error
2008-08-31T18:27:34+00:00 [VCD:xxxx:1001:Info] VCM initialized
2008-08-31T18:27:43+00:00 [FC:3:4019:Major] FC Module state NO_COMM : Cannot communicate with component
2008-08-31T18:27:45+00:00 [FC:4:4019:Major] FC Module state NO_COMM : Cannot communicate with component
2008-08-31T18:27:47+00:00 [FC:3:4011:Warning] FC Module state UNKNOWN : Component operational state cannot be determined
2008-08-31T18:27:47+00:00 [FC:4:4011:Warning] FC Module state UNKNOWN : Component operational state cannot be determined
2008-08-31T18:27:47+00:00 [VCD:GB8813FYTT:1002:Info] VCM-OA communication up
2008-08-31T18:27:47+00:00 [ENC:GB8813FYTT:2001:Info] Enclosure import started : Name [Unknown]
2008-08-31T18:32:40+00:00 [ENET:1:3010:Info] Enet Module state OK : Component fully operational

Any help would be appreciated.

Mark
14 REPLIES 14
JKytsi
Honored Contributor

Re: C7000 FC-VC Modules dropping

fw version 1.30 was withdrawn, current is 1.31 which made some improvements to 1.30 "features".
Remember to give Kudos to answers! (click the KUDOS star)

You can find me from Twitter @JKytsi
mark_uksouth
Frequent Advisor

Re: C7000 FC-VC Modules dropping

Yes i've seen that 1.31 is out and 2.25 for OA is also out.
I could upgrade these but why is this only affecting the boxes connected to uplinks 1 & 2?
I have also tried dynamic logon balancing ands static but still have the same problem.
mark_uksouth
Frequent Advisor

Re: C7000 FC-VC Modules dropping

Looking at the rev history of 1.31 it does not list 1.30, does anyone know why this was withdrawn?
JKytsi
Honored Contributor

Re: C7000 FC-VC Modules dropping

It had unwanted "features"
Remember to give Kudos to answers! (click the KUDOS star)

You can find me from Twitter @JKytsi
mark_uksouth
Frequent Advisor

Re: C7000 FC-VC Modules dropping

unwanted features as in it doesnt work properly?
I find it strange how it is not even on the revision history of 1.31! surely it should be as it was a release and as such has been downloaded and installed on systems.
JKytsi
Honored Contributor

Re: C7000 FC-VC Modules dropping

I don't know really what happened, but now with latest OA & VC firmwares everything works.
Remember to give Kudos to answers! (click the KUDOS star)

You can find me from Twitter @JKytsi
mark_uksouth
Frequent Advisor

Re: C7000 FC-VC Modules dropping

Ok last night we had the same problem with the c7000 and HBA's dropping off the FC VC.

2008-09-03T18:00:29+00:00 [ENC:Unknown:2013:Critical] Enclosure state FAILED : Component is not operational due to an error
2008-09-03T18:00:29+00:00 [VCD:CRW-SAP_HR:1022:Critical] Domain state FAILED : Component is not operational due to an error
2008-09-03T18:00:29+00:00 [VCD:CRW-SAP_HR:1001:Info] VCM initialized
2008-09-03T18:00:38+00:00 [FC:3:4019:Major] FC Module state NO_COMM : Cannot communicate with component
2008-09-03T18:00:40+00:00 [FC:4:4019:Major] FC Module state NO_COMM : Cannot communicate with component
2008-09-03T18:00:42+00:00 [FC:3:4011:Warning] FC Module state UNKNOWN : Component operational state cannot be determined
2008-09-03T18:00:42+00:00 [FC:4:4011:Warning] FC Module state UNKNOWN : Component operational state cannot be determined
2008-09-03T18:00:42+00:00 [VCD:GB8813FYTT:1002:Info] VCM-OA communication up


I took this oportunity to update the OA and VC firmwares to 2.25 & 1.31.
There doesnt seem to be any changes in these and they took almost 2 hrs to complete (fully populated c7000 chassis).

This morning the same thing happened again to profiles connected to ports 1 & 2 of both FC VC's.
This really is getting annoying, why does this not happen to the profiles connected to port 4?

All the settings are identical for all ports so there should be no issues of mis-configuration as port 4 never has a problem.

Also the web admin is so painfully slow, its a complete joke! It takes 15 mins to update after the profiles are connected!
I've never had any issues with the old p-class and the brocade switches and gbe2.

I've now moved profiles from port 1 to 3 to see if this makes any difference.

Anyone have any idea's or similar problems?
We are now thinking of canning VC and going back to standard switches.
kums
New Member

Re: C7000 FC-VC Modules dropping

HI Mark

Please do a failover of the OA.
cjb_1
Trusted Contributor

Re: C7000 FC-VC Modules dropping

Mark

Seen similar problems. Mainly with bays losing profiles and modules rebooting themselves (fixed in vc 1.31). Going to versions 2.25 OA and 1.31 VC helped but did notice some strange activity (missing server details in interconnect bay details, ip addresses not being assigned in the ebipa) until we reset all OA, VC enet modules and the FC modules.

This also sorted the painful webadmin. VC connect gui hung everytime we tried to change one piece of config!

It's also worth going through the enhancements and release notes for the firmware. Just found some completely new functionality for VC FC exists in v1.31 (dynamic san fabric). Again worth a read of the latest (july 08) VC user guide.