vcsu update failed, error health of primary or standby module is not ok

[louisji2008]

I have a new C7000 with 2 vc flex-10/10d and 2 vc-fc 8Gb, after setup network I found vc status is degraded in vcm, the reason could be the 2 vc-fc fw, it shows the 2 module fw is incompitable.
The vc-fc fw is 1.20.0
The vc flex-10 fw is 4.20
So I tried to update the vc-fc using vcsu 1.9.0.

In OA everything is well, I checked the show all output and even in the firmware summary screen there is no any error.
Waiting for your advisor and help.

[Torsten.]

What version do you want to install?

Please post

vcsu -a healthcheck

Do you have the 20 or 24 port VC FC module?

VC version 4.20 brings firmware 2.01 for 24 port,  1.11 for 20 port VC FC 8Gb modules.

There is NO version 1.20 for 8Gb modules.

From the release notes:

[louisji2008]

The content is post in below

[Torsten.]

The screen output would be better.

If you are able to read this, let me know. I can't, sorry.

[Torsten.]

As said, I cannot see the details, but you should do the upgrade using the force "health" option.

[louisji2008]

Sorry for the above content's format. Here is  show firmware summary,  show health and show interconnect status output

>SHOW FIRMWARE SUMMARY

Enclosure Firmware Summary

Onboard Administrator Firmware Information

Bay Model                                             Firmware Version

--- -------------------------------------------------- ----------------

1   BladeSystem c7000 DDR2 Onboard Administrator with KVM 4.22

2   OA Absent                                          

......

Interconnect Firmware Information

Bay Device Model                 Firmware Version

--- ---------------------------- ----------------

1 HP VC Flex-10/10D Module     4.20

2 HP VC Flex-10/10D Module     4.20

3 HP VC 8Gb 20-Port FC Module 1.40

4 HP VC 8Gb 20-Port FC Module 1.40

>SHOW HEALTH

Enclosure Health:

         Enclosure: OK

         Power Subsystem: OK

         Cooling Subsystem: OK

Blade Health:

Bay Status               Problem                           Corrective Action

--- -------------------- ---------------------------------- --------------------

1 OK                  

2 OK                  

3 OK                  

4 OK                  

5 Absent              

6 Absent              

7 Absent              

8 Absent              

9 Subsumed            

10 Subsumed            

11 Subsumed            

12 Subsumed            

13 Absent              

14 Absent              

15 Absent              

16 Absent              

Interconnect Health:

Bay Status               Problem                           Corrective Action

--- -------------------- ---------------------------------- --------------------

1 OK                  

2 OK                  

3 OK                  

4 OK                  

5 Absent              

6 Absent              

7 Absent              

8 Absent              

>SHOW INTERCONNECT STATUS ALL

 Interconnect Module #1 Status:

         Status: OK

         Thermal: OK

         CPU Fault: OK

         Health LED: OK

         UID: Off

         Powered: On

         Diagnostic Status:

                  Internal Data                           OK

                  Management Processor                     OK

                  Thermal Warning                         OK

                  Thermal Danger                           OK

                  I/O Configuration                      OK

                  Power                                   OK

                  Device Failure                           OK

                  Device Degraded                         OK

Interconnect Module #2 Status:

         Status: OK

         Thermal: OK

         CPU Fault: OK

         Health LED: OK

         UID: Off

         Powered: On

         Diagnostic Status:

                  Internal Data                           OK

                  Management Processor                     OK

                  Thermal Warning                         OK

                  Thermal Danger                           OK

                  I/O Configuration                      OK

                  Power                                   OK

                  Device Failure                           OK

                  Device Degraded                         OK

Interconnect Module #3 Status:

         Status: OK

         Thermal: OK

         CPU Fault: OK

         Health LED: OK

         UID: Off

         Powered: On

         Diagnostic Status:

                  Internal Data                           OK

                  Management Processor                     OK

                  Thermal Warning                         OK

                  Thermal Danger                           OK

                  I/O Configuration                       OK

                  Power                                   OK

                  Device Failure                           OK

                  Device Degraded                         OK

Interconnect Module #4 Status:

         Status: OK

         Thermal: OK

         CPU Fault: OK

         Health LED: OK

         UID: Off

         Powered: On

         Diagnostic Status:

                  Internal Data                           OK

                  Management Processor                     OK

                  Thermal Warning                         OK

                  Thermal Danger                           OK

                  I/O Configuration                       OK

                  Power                                  OK

                  Device Failure                           OK

                  Device Degraded                         OK

[louisji2008]

Sorry, I'm quit new to the vcsu. Besides upgrade using the force "health" option, is there any other method to update the VC FC 8Gb firmware? For Example, if the smart update manager can be used?

[Torsten. · Accepted solution]

SUM is basically doing the same.

Run VCSU interactive, answer the question about force options.

"health" means, do the update even if the current status is degraded, not healthy (select this!)

"version" means, do the update on a module that already has this version  (why not...)

Select health and it should work.

[louisji2008]

Do you think sum will be useful if update still failed with health option ?

I found some error in vcsu log, please have a look if some trail can be found:

……

[Thu Dec 12 19:16:30 2019] (2688488)[VcsuCore.cpp:1043] : This is primary enclosure

[Thu Dec 12 19:16:30 2019] (2688488)[VcsuInit.cpp:750] : OA VC Mode is set

[Thu Dec 12 19:16:31 2019] (2688488)[VcsuCore.cpp:1866] : Primary VCM Found at : 192.168.0.23

[Thu Dec 12 19:16:43 2019] (2688488)[OaWrap.cpp:1438] : OA Soap Call failed with error 404

[Thu Dec 12 19:16:43 2019] (2688488)[VcmWrap.cpp:1732] : VCM Firmware Version: 4.20

[Thu Dec 12 19:16:43 2019] (2688488)[VcsuCore.cpp:1259] : VCM IP Address: 192.168.0.23

[Thu Dec 12 19:16:43 2019] (2688488)[VcsuCore.cpp:1260] : VCM Firmware Version: 4.20

[Thu Dec 12 19:16:43 2019] (2688488)[VcsuCore.cpp:1029] : Domain configuration is present

[Thu Dec 12 19:16:43 2019] (2688488)[VcsuCore.cpp:1043] : This is primary enclosure

[Thu Dec 12 19:16:44 2019] (2688488)[VcsuInit.cpp:811] : VCM user authenticated successfully

[Thu Dec 12 19:16:44 2019] (2688488)[VcmWrap.cpp:1732] : VCM Firmware Version: 4.20

[Thu Dec 12 19:16:46 2019] (2688488)[VcmWrap.cpp:901] : Number of enclosures in the domain: 1

[Thu Dec 12 19:16:48 2019] (2688488)[VcsuCore.cpp:1866] : Primary VCM Found at : 192.168.0.23

[Thu Dec 12 19:16:48 2019] (2688488)[VcsuCore.cpp:5531] : VCM is in Unlocked mode

[Thu Dec 12 19:16:48 2019] (2688488)[VcsuCore.cpp:5361] : User has domain privileges

[Thu Dec 12 19:16:48 2019] (2688488)[VcmWrap.cpp:1732] : VCM Firmware Version: 4.20

[Thu Dec 12 19:16:49 2019] (2688488)[VcmWrap.cpp:683] : User has the privilege to update the firmware

[Thu Dec 12 19:16:49 2019] (2688488)[VcmWrap.cpp:688] : User has the privilege to save config

[Thu Dec 12 19:16:49 2019] (2688488)[VcmWrap.cpp:693] : User has the privilege to save support information

[Thu Dec 12 19:16:49 2019] (2688488)[VcmWrap.cpp:1732] : VCM Firmware Version: 4.20

[Thu Dec 12 19:16:50 2019] (2688488)[VcsuSsh.cpp:143] : Connection established to 192.168.0.23

[Thu Dec 12 19:16:50 2019] (2688488)[VcsuSsh.cpp:163] : Successfully established session to 192.168.0.23

[Thu Dec 12 19:16:51 2019] (2688488)[Extract.cpp:334] : Can not open Config\./@LongLink with errorcode as Permission denied

[Thu Dec 12 19:16:51 2019] (2688488)[VcsuCore.cpp:6056] : Extraction of VCM config failed.

[Thu Dec 12 19:16:51 2019] (2688488)[VcsuCoreReal.cpp:383] : Failed to download and extract the config file

[Thu Dec 12 19:16:52 2019] (2688488)[VcsuInit.cpp:911] : Failed to read generation numbers from VCM

[Thu Dec 12 19:16:52 2019] (2688488)[VcsuCore.cpp:1866] : Primary VCM Found at : 192.168.0.23

[Thu Dec 12 19:16:58 2019] (2688488)[VcsuCore.cpp:1866] : Primary VCM Found at : 192.168.0.23

[Thu Dec 12 19:16:58 2019] (2688488)[VcsuCore.cpp:1866] : Primary VCM Found at : 192.168.0.23

[Thu Dec 12 19:16:59 2019] (2688488)[VcmWrap.cpp:763] : Domain has Proper Stacking Connection

[Thu Dec 12 19:16:59 2019] (2688488)[VcmWrap.cpp:774] : Domain has Proper Stacking Redundancy

[Thu Dec 12 19:17:00 2019] (2688488)[VcsuFirmware.cpp:679] : Verifying package signature...

[Thu Dec 12 19:17:28 2019] (2688488)[VcsuFirmware.cpp:697] : Extracting package

[Thu Dec 12 19:18:18 2019] (2688488)[Extract.cpp:125] : Done extracting PackageDesc.xml from VCpackage.tar

[Thu Dec 12 19:18:23 2019] (2688488)[Extract.cpp:125] : Done extracting vcenet.img from VCpackage.tar

[Thu Dec 12 19:18:32 2019] (2688488)[Extract.cpp:125] : Done extracting vcenetfcoe.img from VCpackage.tar

[Thu Dec 12 19:18:46 2019] (2688488)[Extract.cpp:125] : Done extracting vcg2enet.img from VCpackage.tar

[Thu Dec 12 19:19:05 2019] (2688488)[Extract.cpp:125] : Done extracting vcg2enetfcoe.img from VCpackage.tar

[Thu Dec 12 19:19:18 2019] (2688488)[Extract.cpp:125] : Done extracting vcfc.img from VCpackage.tar

[Thu Dec 12 19:19:23 2019] (2688488)[Extract.cpp:125] : Done extracting vcfc2.img from VCpackage.tar

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:709] : Parsing package information

[Thu Dec 12 19:19:35 2019] (2688488)[Extract.cpp:125] : Done extracting PackageDesc.xml from VCpackage.tar

[Thu Dec 12 19:19:35 2019] (2688488)[Extract.cpp:125] : Done extracting ReleaseNotes.txt from VCpackage.tar

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:324] : FIPSCompliant tag is not found

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:324] : FIPSCompliant tag is not found

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:324] : FIPSCompliant tag is not found

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:361] : Information for module type VC-FC3 is not available in this package

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:558] : xml_get_module_Info VC-FC3 failed for PackageDesc.xml

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:324] : FIPSCompliant tag is not found

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:324] : FIPSCompliant tag is not found

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:324] : FIPSCompliant tag is not found

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:769] : ReleaseNotes.htm saved in current directory

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:7434] : Domain is not in FIPS mode

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:7434] : Domain is not in FIPS mode

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:7434] : Domain is not in FIPS mode

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:7434] : Domain is not in FIPS mode

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:7434] : Domain is not in FIPS mode

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:2378] : Checking state of any incompleted firmware updates.

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuSsh.cpp:143] : Connection established to 192.168.0.23

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuSsh.cpp:163] : Successfully established session to 192.168.0.23

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:2471] : No incomplete firmware updates detected.

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:1967] : Module in bay 1 is not powered on

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:2097] : Current firmware revision on 1.400000 of VC-FC module in bay: 3 supports generated FTP port feature.

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:2097] : Current firmware revision on 1.400000 of VC-FC module in bay: 4 supports generated FTP port feature.

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuFirmware.cpp:912] : Unsupported module type passed

[Thu Dec 12 19:19:35 2019] (2688488)[VcsuCore.cpp:1866] : Primary VCM Found at : 192.168.0.23

[Thu Dec 12 19:19:35 2019] (2688488)[VcmWrap.cpp:1732] : VCM Firmware Version: 4.20

[Thu Dec 12 19:19:36 2019] (2688488)[VcsuCore.cpp:7031] : The health of primary or standby module is not Ok. The update cannot proceed

[Thu Dec 12 19:19:36 2019] (2688488)[VcsuCore.cpp:6589] : Failed to update checkpoint status

[Thu Dec 12 19:19:36 2019] (2688488)[GetModuleUpdateInfo.cpp:167] : Failed to get the modules information of the enclosure

[Thu Dec 12 19:19:38 2019] (2688488)[OaWrap.cpp:793] : User successfully deleted

[Torsten.]

This could be the show stopper:

Module in bay 1 is not powered on

SUM is using the same vcsu tool.

You should use the latest and greatest version of vcsu.

I currently have vcsu-1.14.0-x86.msi

https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_5e6075b5c9b246f8810aa0c6b3

Try with "Run As Administrator ..."

[louisji2008]

Thank you Torsten, you are so helpful!

I will try to update in next week. Have a nice weekend!

[louisji2008]

Torsten,
Thank you! With your advise, the problem is fixed.
I did a healthcheck, nothing error found.
Then I run vcsu as administrator, and update with health option.
Thank you again!

[louisji2008]

I have another question about configuring vc network.
If you have time, please have a look:
https://community.hpe.com/t5/BladeSystem-Virtual-Connect/How-to-configure-VC-network-if-I-need-bond-two-nic-with-active/td-p/7073424#.XfrJFUK-vqA