SimpliVity cluster commit query

[jlangmead]

Hi

I'm curious to understand why a successful SimpliVity cluster upgrade will fail to commit if there is a faulty node in a completely different cluster within the same federation?

We have several clusters within a single federation and will eventually look to update all the clusters to 5.2 and restire some of the clusters altogether. However, one of the test clusters currently has a faulty node due to a hardware issue. We have successfully upgraded one of the clusters to 5.2 but the commit operation fails due to the faulty node in our test cluster. I'm curious to understand why this is preventing the commit from succeeding and if there is a workaround other than fixing or evicting the faulty node.

Many thanks in advance

[Joshua610]

Thank you for reaching out. 

I see you are curious as to why the commit fails after a successful upgrade with a faulty node within another cluster within the same federation. Without proper troubleshooting I can only provide some insights in this direction. 

1: Whenever I work a live upgrade case I like to monitor what's all going on & can do so while running these 2 commands, 1 in Powershell & 1 on the OVC being upgraded in order to see an issue while or before it happens. 

VIA Powershell: 
Get-Content C:\Users\svtuser\Appdata\local\upgrademanager\orchastartor<current date>.log 

VIA OVC: 
tail -f /var/log/svt-upgrade.log

This will help identifiy any issues preventing progress. 

However, I do want to provide you access to the Upgrade guide where I pulled the following information from as well. 

Link for HPE OmniStack 5.2.0 for vSphere Upgrade Guide: 

https://support.hpe.com/hpesc/public/docDisplay?docId=sd00005177en_us&page=index.html

Some need to know info: 
"You do not need to commit the upgrade to complete the process for ESXi or HPE host firmware upgrades unless you are upgrading ESXi from 6.x to 7.x. In that case you must first commit the HPE OmniStack upgrade to provide support for this new feature before you can use HPE SimpliVity Upgrade Manager to update the ESXi version."

Based on prerequisites: 
"If you are upgrading a centrally-managed federation, verify that all of the management virtual appliances are running and healthy. For more information, refer to Verify Management Virtual Appliance health through vSphere Client."

Found here: 

https://support.hpe.com/hpesc/public/docDisplay?docId=sd00005177en_us&page=GUID-4AA963F4-52F9-4141-82F7-A977F0A8E38F.html

Let me know if you need any further insight and or can provide more details that would assist in my direction, thank you and have a great weekend. 

[Joshua610]

I am also investigating for know issues/workarounds as well. I will have something by end of day-Monday, thank you. 

[Joshua610 · Accepted solution]

Good day friend, 

As promised: 

"We have successfully upgraded one of the clusters to 5.2 but the commit operation fails due to the faulty node in our test cluster. I'm curious to understand why this is preventing the commit from succeeding and if there is a workaround other than fixing or evicting the faulty node."
= 
"If you are upgrading a centrally-managed federation, verify that all of the management virtual appliances are running and healthy. For more information, refer to Verify Management Virtual Appliance health through vSphere Client."

&&

"NOTE
If you are upgrading from 3.7.8 or earlier, you must first upgrade all clusters in the federation and then perform federation commit using the Commit button at the bottom of Upgrade Manager. If you are upgrading from 3.7.10 Update 1 or later, you can upgrade each cluster individually and perform cluster commit using the Commit option available in the hamburger menu next to the cluster in Upgrade Manager."

Also Note:

"You cannot access new features in the HPE OmniStack software upgrade until you commit it to all the HPE OmniStack hosts in the cluster."

This situation willl be addressed in future versions of SimpliVity for I can see this is being tested currently (I do not have a definitve answer on release date or version # currently). 

[5.2.0 Guide where information is being pulled from.] 

https://support.hpe.com/hpesc/public/docDisplay?docId=sd00005177en_us&page=GUID-4AA963F4-52F9-4141-82F7-A977F0A8E38F.html

[jlangmead]

Many thanks for the prompt reply

I felt this was 'expected behaviour' rather than anything that would need further troubleshooting as the information provided in the error code was pretty clear as to why the commit was being denied. It was just unclear as to why this was necessary. We fixed the issue with the faulty node and the commit succeeded - so no big problem for us really.

I'm glad to hear this type of scenario will be addressed in a future release.

thanks

Jon