Virtual Connect Manager-Virtual Connect Enterprise Manager issues after Backplane replacement

[chuckk281]

Arnout had a customer issue that he was looking for help:

***************************************

Hello experts,

I am stuck at the moment due to a lack of in depth knowledge…

Situation:

-        3 enclosures per domain

-        VCEM used for multiple domains

-        1 enclosure has apparently bad midplane

-        Midplane replaced, serial number has been set back, but apparently all modules were in the enclosure when the spare serial number was still there

-        VCEM gives red on everything in that enclosure.

Is it possible to get this corrected?  I don’t know if the customer has a backup of the domain, but if he has, where should he do the restore (since the backup contains info about all enclosures in that domain).

And what if he has no backup?

Thanks for some quick answers.

**************************************

Monty replied:

**********************************

You have encountered the problem in the service advisory c01297654.  This issue affects both Virtual Connect and VCEM domains.

If the advisory had been followed – only the OA modules, fans and power supplies would have been in the enclosure after the midplane replacement – so that the midplane SN, PID and PDU type could be manual updated using the OA CLI. 

Then all the interconnects and blades could be inserted and there would be no loss of the VCM or VCEM domain.

Virtual Connect Manager keys the domain to the enclosure serial number as reported by the Onboard Administrator – and after the midplane is replaced the OA reports the replaced midplane serial number and VCM disables the domain and all server ports.

When VCEM is managing the domain, the same dependency exists – VCEM keys each enclosure in the domain with the enclosure serial number – obtained from the OA and SIM.

To repair this situation you will need to:

• Remove the enclosure from the VCEM domain
• Turn off the primary VC interconnect modules in bays 1 and 2
• Turn off all the server blades
• Update the enclosure SN, PID and PDU_type using the OA CLI and the Administrator local account credentials
• Turn on all the VC interconnect modules in bays 1 and 2 (they will now obtain the correct enclosure serial number
• Log into VCEM and import the enclosure
• At this point you may be able to restore the previously saved VCEM configuration which will restore the profiles to the blades in this enclosure
• Now turn all the server blades back on in this enclosure.

We have been working the last couple of weeks on updating the documentation to be included in each midplane assembly kit to include the steps to prevent loss of the VC or VCEM domain.

******************************************

Have you had this problem? What did you do?