MSA Controller B Heartbeat Lost - Seeking Help

[XFreitas]

Hello everyone,

I've been experiencing a problem with one of the controllers on my company's MSA 2040 Storage. Specifically, Controller B keeps going down with the error message "Heartbeat lost."

To resolve the issue, I have been detaching the controller, cleaning the connectors, and reinstalling it. However, I noticed the following message in the Event Log:

"Firmware versions do not match those in the firmware bundle. (controller: B, mismatched components: SC, EC)
Additional Information:
When the Management Controller (MC) is restarted, firmware versions that are currently installed are compared against those in the bundle that was most recently installed. When firmware is updated, it is important that all components are successfully updated or the system may not work correctly. Components checked include the CPLD, Expander Controller (EC), Storage Controller (SC), and MC.

Recommended Action:
- Reinstall the firmware bundle."

I'm seeking advice on whether it is reliable and safe to reinstall the firmware bundle as recommended. I have never done this before and I am concerned about potential data loss.

Any help or suggestions would be greatly appreciated.

Thank you in advance!

[ArunKKR]

Hi,

Could you share the output of below commands?

show versions detail

show pools

show disk-groups

[XFreitas]

Sure thing, here are the outputs:

# show versions detail

Controller A Versions

---------------------

Storage Controller CPU Type: Gladden 1300MHz

Bundle Version: GL101R001

Build Date: Fri Aug 16 14:57:03 MDT 2013

Storage Controller Code Version: GLS105R04-01

Storage Controller Code Baselevel: GLS105R04-01

Memory Controller FPGA Code Version:

Storage Controller Loader Code Version: 27.015

CAPI Version: 3.19

Management Controller Code Version: GLM101R001-01

Management Controller Loader Code Version: 5.79.16319

Expander Controller Code Version: 3117

CPLD Code Version: 42

Hardware Version: 3.2

Host Interface Module Version: 6

Host Interface Module Model: 6

Backplane Type: 7

Host Interface Hardware (Chip) Version: 2

Disk Interface Hardware (Chip) Version: 3

SC Boot Memory Reference Code Version: 1.2.1

Controller B Versions

---------------------

Storage Controller CPU Type: Gladden 1300MHz

Bundle Version: GL101R001

Build Date: Fri Aug 16 14:57:03 MDT 2013

Storage Controller Code Version: GLS105R04-01

Storage Controller Code Baselevel: GLS105R04-01

Memory Controller FPGA Code Version:

Storage Controller Loader Code Version: 27.015

CAPI Version: 3.19

Management Controller Code Version: GLM101R001-01

Management Controller Loader Code Version: 5.79.16319

Expander Controller Code Version: 3117

CPLD Code Version: 28

Hardware Version: 4.2

Host Interface Module Version: 6

Host Interface Module Model: 6

Backplane Type: 7

Host Interface Hardware (Chip) Version: 2

Disk Interface Hardware (Chip) Version: 3

SC Boot Memory Reference Code Version: 1.2.1

Success: Command completed successfully. (2023-10-09 16:23:58)

# show pools

Error: The command was not recognized. (2023-10-09 16:24:51)

# show disk-groups

Error: The command was not recognized. (2023-10-09 16:25:21)

Just the first one worked properly

[ArunKKR]

Hi, Please check whether the below command works:

show vdisks

[XFreitas]

Hi,

It worked, here is the output:

# show vdisks
Name  Size     Free    Own Pref   RAID   Disks Spr Chk  Status Jobs      Job%      Serial Number                    Drive Spin Down        Spin Down Delay       Health     Health Reason Health Recommendation
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vd01  4196.5GB 347.3GB A   A      RAID6  9     0   512k FTOL                       **removed** Disabled               0                     OK
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully. (2023-10-10 07:27:08)

[Moderator note: Post-edited and removed the confidential information.]

[ArunKKR · Accepted solution]

Hi, MSA has single vdisk owned by controller A. Controller B is currently in stand by mode. There should be minimal risk if you reflash the controller B firmware. Please note that GL101 is  the initial version of firmware. This version does not seem to be available for download. Any firmware download requires unit to be under warranty/extended warranty. Please check whether partner firmware update option is enabled using the below command:   show advanced-settings If partner firmware update option is enabled this should ideally sync the firmware between controllers automatically.

[XFreitas]

After looking at this output, I think it's better to give up.
The MSA was set up back in 2013, even before I joined the team, so my chance to update it has come and gone.
Thanks a lot for your help!