SimpliVity and VMware article 88856

[CadenLange]

Greetings

I would like to find out more about the following VMware article and the impact it may have with Simplivity. The article is this one 3rd party Hyper Converged Infrastructure setups experience a soft lock up and goes unresponsive indefinitely (88856) (vmware.com) and I first saw it mentioned in this post:

HPE SimpliVity OmniStack 4.2.0 and Firmware SVTSP-... - Page 2 - Hewlett Packard Enterprise Community

It was added by a vistor as a comment to the above post but has had no further resposnes - I thought I'd start a new thread as I think I've experienced this very issue but didn't really consider if it had any direct relation to the VMware article - but I feel it might so wanted to get more info.

Essentially, during an upgrade from an older version of ESXi (pre 7.0 U3i) all the vm's went offline because the NFS datastores became inaccessible.

Whereas I accept this is a VMware issue that has now been fixed, I wanted to ask if it's necessary to apply the workaround suggested by VMware to avoid this happening (eg  esxcfg-advcfg --set-kernel "FALSE" vtdEnableIntrVirt)

Or if just completing the firmware upgrades at the end of the upgrade steps is sufficient  - this is what the SimpliVity forum post says is all that is required here HPE SimpliVity OmniStack 4.2.0 and Firmware SVTSP-... - Page 2 - Hewlett Packard Enterprise Community

"

Just for you to know - if somebody is wondering like me about the change of the preferred order of installations for an update from 4.x.x to 4.2 (firmware should be deployed as the last part of the cycle):

--- advised customer to follow the procedure offered from the Upgrade planner, because of the following VMware Article: https://kb.vmware.com/s/article/88856?lang=en_US which is:
update arbiter -> update vSphere plugin -> update VCSA -> update OVC -> update ESXi -> update HPE host firmware
--- if we have an ESXi Version that includes this mentioned bug, firmware update can cause APD.

This is also suggestet by the HPE Simplivity Update Planner."

many thanks in advance

[support_s]

System recommended content:

1. HPE SimpliVity - After ESXi Restart SimpliVity Datastores Are Not Accessible

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Sanika]

Hi CadenLange 

Based on my understanding of the information you provided, it sounds like you experienced an issue during an ESXi upgrade where VMs went offline due to NFS datastores becoming inaccessible. This was a known issue in older ESXi versions that has since been fixed in later releases.
VMware suggested a workaround by running "esxcfg-advcfg --set-kernel "FALSE" vtdEnableIntrVirt".
However, according to the SimpliVity forum post you referenced, simply completing the firmware upgrades at the end of the upgrade process should be sufficient to address this issue.

In my opinion, applying the workaround is not necessary if you have already completed the firmware upgrades at the end of the upgrade steps. This is confirmed by the SimpliVity forum post you mentioned. The firmware upgrades address the root cause of the issue, making the workaround redundant. However, it might have performance implications and should only be used if firmware upgrades are not possible.

If you haven't done the firmware upgrades yet, then applying the workaround can be a temporary measure to prevent the datastores from becoming inaccessible during the upgrade. However, it's crucial to prioritize the firmware upgrades as the permanent solution.

The choice is ultimately yours. In order to maintain a long-term and reliable solution, I recommend prioritizing firmware upgrades.

Additional resources that might be helpful are listed below:

https://communities.vmware.com/t5/VMware-vCenter-Discussions/NFS-Datastore-inaccessible/td-p/2286799
https://kb.vmware.com/s/article/1003967
https://support.hpe.com/hpesc/public/docDisplay?docId=a00123775en_us&docLocale=en_US

If you have further doubts regarding this, I would recommend you to contact HPE support team for more assistance.

Hope this information helps!

Regards,
Sanika.

If you feel this was helpful, please click the KUDOS thumb below. Also consider marking this as an "Accepted Solution" , if the post has helped to solve your issue.

[CadenLange]

Hi Sanika

Thanks for taking the time to answer my query.

Your understanding is correct on the situation and I was looking for confirmation that leaving the firmware upgrades to the end of the upagre activity is a valid workaround - this is for future upgrades I have planned on other clusters.

I have several other clusters remaining to be upgraded and these will be from SimpliVity version 4.1.0 U1, ESXi 7.0 U2 and SVT 2023-0110.01.   I plan to bring all these clusters up to the latest supported SimpliVity, ESXi and firmware version.

I use the recommended approach of using the SimpliVity Upgrade Manager to complete the upgrades in a non-disruptive manner. As Upgrade Manager will now do the SimpliViuty software, ESXi version AND firmware all together, I am wondering if the correct procedure would be to allow Upgrade Manager to ONLY do the SimpliVity software (to 4.2.0) and the ESXi version to 7.0.U3o BUT to leave the firmware as 'No Action' to avoid this issue. Once the OVC and ESXi upgrades have completed I can then upgrade the firmware (to SVT 2023-1130) either manually by booting into the iSO or again via Upgrade Manager (just doing the firmware update as the OVC and ESXi are already complete).

Are you able to advise if that approach would avoid the issue in the VMware articale?

many thanks in advance.

[gustenar · Accepted solution]

You can leave the firmware updates to be the end as long as the current firmware package you have is supported or compatible with the latest software version. Check the HPE Simplivity Interoperability Guide to see if the firmware package you have is compatible with 4.2.0. In any case, the Upgrade Manager will do the software first, ESXi second, and will leave the firmware as the last step of the upgrades. 

[CadenLange]

Thank you both for your input - that makes sense and I will follow your advice.

kind regards

C

[Sunitha_Mod]

Hello @CadenLange,

That's awesome! 

We are extremely glad to know your concern has been addressed.