Virtual Connect Domain failure

[Steve MacKenzie]

Hi,
I have the following issue, whenever I try and edit or delete a particular server profile the entire VC domain resets with the following error:
Enclosure state FAILED : All Enet modules not OK or DEGRADED
Critical] Domain state FAILED : 1+ enclosures not OK or DEGRADED
VCM initialized

All blades lose network connectivity until the modules re-initalize.

We are running:
OA version 3.21
VC version 3.15
All Blades on BIOS l24 09/02/2010

Any help appreciated.

[dchmax]

I've not seen this before. You could try failing the active vc domain module to the standby and make the same changes. Hopefully it can be narrowed down to a specific module issue. Support will more then likely walk you through reseating all the modules and the backplanes. What type of modules do you have in interconnect bay 1 and 2?

[Steve MacKenzie]

We have - HP VC Flex-10 Enet Modules.
We have logged a support call and they are still going through the logs.

[dchmax]

Post what you find out. I'm about to upgrade the firmware in our test chassis.

[anthony.bucci]

Have you figured out what is wrong here? I have the same issue right now.

OA version 3.21
VC version 3.15

VC-ENet gives a no communication error.
VC-FC seems to be working just fine.

Thanks!

[Steve MacKenzie]

This is the response for the support team:

"recommended that the only way to resolve this is to deleting and recreating the Domain.
This issue could be caused due to corruption in the Virtual Connect Domain Configuration. Hence to resolve the VCD crashing issue every time a profile is edited or deleted, is to delete the existing VCD configuration and create a new Domain."

Which isn't good when I'm running crtical production servers in the enclosure.

I have escalated the issue, but have yet to receive another response.

[anthony.bucci]

Thanks for the update...
I've done that and it does not seem to help either...

very bad answer on the part of HP.

Can't wait to hear what the next level has to say.

Thanks again.

[ChrisTan_1]

Hi Steve,

are you using any of the new features in ver 3.15? I noticed you are still using Flex-10 and not FlexFabric modules so VC firmware ver 3.10 should be sufficient.

Force a VC firmware downgrade to ver 3.10?

[Steve MacKenzie]

The issue started while we were running 3.10
We were advised we needed to upgrade our firmware to the revisions mentioned in my 1st post.
After upgrading the issue remains.

Regards,
Steve

[DerekS_1]

I had a similar problem in December when we did VC/OA firmware upgrades. Two linked chassis, and VC died on us such that all blades in both chassis lost network connectivity. Critical production servers were taken offline and I was not at all happy.

After several hours of being on the phone with HP, I gave up. I'm not sure what finally cured the problem, but I ended up de-assigning and assigning a profile to a blade then things got back in sync and the servers came back online.

I'm not at all a fan of VC, and will avoid future purchases of this technology as my faith in its reliability is nearly gone.