Upgrading firmware in a Multi-Enclosure Stacking (MES) Domain

[chuckk281]

Mary was looking for some MES info:

****************

  I have a major customer that has 3 enclosures stacked together. They are running old FW and we need to upgrade FW to VCM v3.51. I know as it stands we need to upgrade all enclosures at once, but that is a major issue for this customer.

Do we have any other options. Such as pull one chasis out and upgrade it, pull another chasis out and upgrade it, do the main enclosure and once they are all upgraded restack them back together?

******************

Multi replies:

***************

From Chris:

VC firmware in a Multi-Enclosure Stack MUST be upgraded at the same time.  You cannot stage VC firmware deployment.

From Hoa:

I have updated 3 enclosures in MES configuration, simulated live pre-production with no problem.

1.       Perform “manual update” on odd or  even bays, opposite from where VCM is running from.

2.       After successful update, manually activate those bays to newer firmware.

3.       VCM forced failover to the updated side.

4.       Repeat step 1 and 2 for the remainder bays.

From Andrea:  

As far as I experienced, when you upgrade the VC FW of a MultiEnc Domain you point to the OA of the enc0 and upgrade with VCSU all enclosures one at a time.

I've tried the alternative method of installing the firmware instead of install&activate, to do one enclosure at a time (this can be done with -oe manual and -of manual options in VCSU and then reboot every VC module) but since Fw 3.xx you still have some outage when activating the firmware of the last two modules (primary and stdby ones) and the outage is widespread on all stacked enclosures.

So from what i've seen, there's no point in doing fw install only and manual vc reboot for firmware activation.

However i'm always in the hope that someone will eventually say that i'm wrong...

From Hoa:

Our goal was to avoid any outage and manual activation worked for us, activate odd or even bays opposites from VCM bay.  We watch ping -t as activation took place, at most less than 4 seconds but not severe for VM machines and Oracle clusters to drop.  Fabric outage (excerpts from VC 3.51 Release Notes) to be expected in some cases but wide spread outage I think has deeper root in something else.

VCSU 1.60 Release Notes also mentioning manual activation as work around for problem mentioned by Andrea on Page 10.

******************

Your experiences? Comments or suggestions?