OneView for VMware vCenter ov4vc not working after OneView 8.9 upgrade

[ndavis_cmm · Accepted solution]

We have made some progress on the case. Kudos to my coworkers that jumped on the call with HPE Support again today.

Changing the iLO security settings had no impact on our issues - high security (assuming you have the credentials in isut properly) or production, it made no difference. 

We have found there is an issue with the OneView database not updating firmware related fields correctly. By changing the firmware baseline option in the server profile to "managed manually" it populates the needed field in the database. 

Steps from our call with HPE today - seems to be an effective workaround, hopefully it will be resolved soon. 

1

log into "your oneview"

3

next, navigate to server profiles

4

for each host listed, select it under server profiles, then Actions > Edit

5

Under Firmware > Firmware Baseline > select the drop down and switch it to “managed manually” then click OK

6

Allow it to refresh in oneview

7

After completing on each host, log into "your vCenter" > navigate to the cluster > select 'updates' tab and click “Check Compliance”

8

All of the hosts should report back their status rather than showing ‘unknown’

9

Back in oneview, select each host > actions > update from template > keep defaults and accept it.

I'm hoping this is helpful for you all as well wait for a long-term solution. 

[Sunitha_Mod]

Hello @ndavis_cmm,

We are glad to hear you found the work around and we appreciate you for keeping us posted. We hope this helps the other users in the Community. 

[robinharmsen]

@Sunitha_Mod any news on a good/final solution?

I am currently trying the last mentioned workaround.. seems ok.
But I am wondering whether we will have issues again during a new update round.

[robinharmsen]

Update... we still not seem to be able to do a cluster remediation with this

[ndavis_cmm]

One thing we have noticed is that you have to make sure all of the hosts in the cluster have had the workaround completed before you try to do anything at the cluster level. If one host fails the compliance check in the cluster the entire cluster fails. 

Not sure if that was your issue, but it is worth mentioning. Thanks! 

[robinharmsen]

Well the compliance check worked.. but the remediation from vLCM did not.
So back to updating one by one which takes a lot of time

[robinharmsen]

Saw, OneView 9.0 has been released.

Hope that fixes stuff.. as we are currently on 60 from 74 servers updated (one by one) we will first finish that.
After that we will try upgrading oneview and see whether full vLCM works again.

[pirx]

As I use the "monitored mode", this workaround will not work for me.

[pirx]

I'd be careful with OV 9.00 and OV4VC 11.6, I guess the combination is not supported. Or it is and not  working like 8.9 + 11.6.

[rkiik]

Updating to 9.00 breaks the updating again, even if you fixed it in 8.90.

Made a script that resets all the server profiles to manual then prompts the update from template, no need to check compliance in vcenter in middle. Seemed to fix for me.

Overall if you are thinking of switching to 8.90 or 9.00, stay away. I would NOT install this in production.

[ndavis_cmm]

Well that is disapointing. 

I am curious if you would be willing to share the script you are using? That might be helpful to others! 

[PriitP]

There is no need to set the server profile back to any SPP after setting it to manual. Just go to vCenter and run check compliance, firmwares will be detected and you can start the remediation. During remediation vLCM will assign the SPP you specified in a cluster image to the server profile and do the updating of firmwares like before.

Quite annoying bug anyway...

[ndavis_cmm]

Significant update from HPE on this issue:

Update from dev ops: (the workaround provided is similar, but doesn't require a reboot)

The issue will be fixed in OneView 9.10 Build 494780.

Following is the workaround,

1. For Managed Mode Servers with an associated Server Profile:

Reapply the profile by selecting the "Reapply" option from the Actions menu.

Alternatively, use the provided REST call:
PATCH https://{appl}/rest/server-profiles/{id}
[{ "op" : "replace",
"path" : "/firmware/reapplyState",
"value" : "ApplyPending"
}]


2. For Monitored Mode Server Hardwares:

Select "Update firmware" from the Actions menu and apply the previously used SPP.

Alternatively, use the following REST call:
PATCH https://{appl}/rest/server-hardware/{id}/firmware/settings
[ { "op": "replace",
"value": {
"baselineUri": "/rest/firmware-drivers//{fwBaselineId}",
"firmwareInstallType": "FirmwareOnly",
"installationPolicy": "LowerThanBaseline"
}
}]

Hopefully we will have a solution available soon! 

[pirx]

2. For Monitored Mode Server Hardwares:

Select "Update firmware" from the Actions menu and apply the previously used SPP.

What to you mean with apply? Do the actual firmware update via OV action? Clearly this solves the issue of a non working update in vLCM... There is only an ok button which starts the update after chosing the baseline. For that the server must be powered off.

....

[ndavis_cmm]

That was just a dump from what the HPE support rep gave us.

We do not run in monitored mode so I'm not sure I can answer accurately. 

[pirx]

Yes, and thanks for that. As I'm using monitored mode, I just reflect what this means..

[Cederberg]

Running a test environment that i recently updated to oneview 9.0 and oneview for vmware vcenter is 1.6. Two esxi DL380 gen10 servers in vcenter 8.0U2. Both server hardwares added to Oneview in Monitoring mode.
After updating to Oneview 9.0 it could not determine compliance and both hosts where status Unknown.
Tried the recent workaround to apply firmware from Oneview. GUI not RestAPI. I selected the SPP i used in vLCM (SPP2024.04.00) and pressed OK. The server needs to be powered off and it took about 5 minutes for it to start the server and figure out that it did not need any firmware updates and it shut down again.
Tried to check compliance when i finished the first of the two hosts but both where status unknown. After i finished repeating the workaround on the second of my two hosts it gave me a Compliant result on the image compliance check. As i only have two test esxservers in Oneview and vcenter i don't know if you need to do the workaround on all servers in oneview or if it's enough to do it on the servers in the cluster you are trying to compliance check.
Not a workaround suitable for anything other than a very small environment but at least it worked.

[ndavis_cmm]

https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00142091en_us&hprpt_id=ALERT_HPE_3069645&jumpid=em_pom8nu6hj_aid-521053889

Official advisory notice from HPE

[RenjuCR]

As mentioned above this is an issue with OneView versions 8.90 and 9.0.

You will not observe this issue with OneView version 8.80 and latest OV4VC version.

This will be fixed in the next version of OneView mostly.

[robinharmsen]

Any news on when me can expect OneView 9.10 ?

[RenjuCR]

As per the current schedule OneView 9.10 is mostly out by 1st week of Sep 2024, assuming no other problems arise during release qualification..