OV4VC 11.3, pre-check failing, server profiles still necessary?

[pirx]

I'm failing in getting OV4VC and vLVM to work as replacement for ILO Amplifier. I read the documentation and release notes again and I'm confused. Is it still necessary to create a server profile in OV to use OV4VC as HSM + vLCM? Documentation descibes the steps to create a server profile, release notes point out that its not needed. 

I have no OV licenses, I use the "Monitor" option not "Managed" for servers. In vLCM image all looks as expected until I start pre-check.

Document - HPE OneView for VMware vCenter 11.3.0 User Guide | HPE Support

Document - HPE OneView for VMware vCenter 11.3.0 Release Notes | HPE Support

"Ability to do firmware update using vSphere Lifecycle Manager without a server profile for both monitored and managed servers"

vLCM log:

HsmService:2023-08-02 15:22:51,965[Dummy-33]hsmService:416 [INFO] Retrieved information of task b723ad42-735c-4178-ac72-4a2fe3584327: {"action":"GET_HOST_INVENTORY","description":"Firmware inventory details of host host-2496701","estimatedTimeRemaining":0,"hosts":["host-2496701"],"id":"b723ad42-735c-4178-ac72-4a2fe3584327","messages":["Error occured during inventory of host-2496701"],"operationStatusCode":500,"progress":100,"startTime":"2023-08-02 15:22:38.422115","status":"FAILED"}
HsmService:2023-08-02 15:22:51,966[Dummy-33]hsmService:758 [INFO] Hsm service result: {'output': '{"action": "GET_HOST_INVENTORY", "description": "Firmware inventory details of host host-2496701", "estimatedTimeRemaining": 0, "hosts": ["host-2496701"], "id": "b723ad42-735c-4178-ac72-4a2fe3584327", "messages": ["Error occured during inventory of host-2496701"], "operationStatusCode": 500, "progress": 100, "startTime": "2023-08-02 15:22:38.422115", "status": "FAILED"}', 'error': None}


HsmService:2023-08-02 15:23:53,052[Dummy-42]hsmService:416 [INFO] Retrieved information of task 994d7ca1-d828-454a-a596-a9ea8f27a199: {"action":"UPDATE_PRE_CHECK","description":"Precheck update for host-2496701","estimatedTimeRemaining":0,"hosts":["host-2496701"],"id":"994d7ca1-d828-454a-a596-a9ea8f27a199","messages":["Failed precheck for host-2496701"],"operationStatusCode":500,"progress":100,"startTime":"2023-08-02 15:23:21.007039","status":"FAILED"}
HsmService:2023-08-02 15:23:53,052[Dummy-42]hsmService:758 [INFO] Hsm service result: {'output': '{"action": "UPDATE_PRE_CHECK", "description": "Precheck update for host-2496701", "estimatedTimeRemaining": 0, "hosts": ["host-2496701"], "id": "994d7ca1-d828-454a-a596-a9ea8f27a199", "messages": ["Failed precheck for host-2496701"], "operationStatusCode": 500, "progress": 100, "startTime": "2023-08-02 15:23:21.007039", "status": "FAILED"}', 'error': None}

I also noticed that vLCM isut state fail, but if I enter the password it does not fix the state but I see that amsd and isut were restarted sucessfully. Both servervices were running before and after, sut is set to AutoDeploy.

2023-08-02T14:17:52.448Z sshd[5838021]: Connection from 10.11.5.76 port 53766
2023-08-02T14:17:52.591Z sshd[5838021]: Accepted keyboard-interactive/pam for root from 10.11.5.76 port 53766 ssh2
2023-08-02T14:17:52.609Z sshd[5838021]: pam_unix(sshd:session): session opened for user root by (uid=0)
2023-08-02T14:17:52.649Z sshd[5838021]: User 'root' running command '/etc/init.d/amsd restart'
2023-08-02T14:18:10.134Z sshd[5838021]: User 'root' running command 'sut -set mode=AutoDeploy'

# sut /status
System Update Manager...................: OS Administrator
Task Status.............................: Bios:N/A; LocalStorage:N/A; Deploy:Idle
Staging Directory.......................: /tmp/stagingdirectory
Baseline URI............................: None
Baseline Version........................: Waiting for staging
Mode of Operation.......................: autodeploy
Polling Interval In Minutes.............: 5
Force Deploy............................: No
Optional Components.....................: None
EnableiLOQueuedUpdates..................: true
User Action Needed......................: To perform firmware update, in OneView, import the server, create, and apply a server profile with baseline. In Compute Ops Management, connect the server to CoM and trigger an Update Firmware action. In iLO Amplifier Pack, import the server and perform Firmware Update.
iLO Security Mode.......................: Production
iLO CAC Smartcard Authentication........: Off
iLO CAC Strict Mode.....................: Off


# /etc/init.d/amsd status
amsd-smarev is running 1690985889 1
amsd-ahsd is running 1690985888 1
amsd-amsd is running 1690985886 1
amsd-smad is running 1690985885 1

But the vLCM image looks good, status shows that only latest ILO firmware is missing from SPP, So I don't understand why the precheck fails then.

[pirx]

I think I can answer at least part of my own question.

"Ability to do firmware update using vSphere Lifecycle Manager without a server profile for both monitored and managed servers"

The release notes seem to be wrong. Or I don't get it. The log clearly states that the task failes becasue the server profile could not be updated. Yes, because there is none.

INFO:HsmServiceLogger:Retrieved information of task 6160951e-1bc3-4159-91b6-391d329f6821: {"action":"POST_IMAGE_UPDATE","description":"PostImage update for host-3047462","estimatedTimeRemaining":0,"hosts":["host-3047462"],"id":"6160951e-1bc3-4159-91b6-391d329f6821","messages":["\"Error occurred while updating the firmware baseline in server profile.\""],"operationStatusCode":500,"progress":100,"startTime":"2023-08-03 09:20:43.395074","status":"FAILED"}

[MV3]

Hello,

If the iSUT is working fine, and you are still getting an error, Can you please try rebooting the OneView for VCenter and check?

Cheers...

[pirx]

I noticed that the state of isut did not really change to AutoDeploy when I issued the command manually or when I triggered it from vCenter. The commans finished successful but the old status was still shown. Even a reboot of the server did not change that. I first had to change it to OnDemand and then to AutoDeploy. After that the pre-check was also ok. Typicially we have clusters with 8-10 hosts and this happend in 2 clusters for  1 host. 

[MV3]

Hello,

You will have to raise a support case with HPE for a detailed analysis of the issue.

Cheers...

[vmuniverse1]

This looks simular to my problem I have now. All of my servers have a red cross in the vLCM Pre-Check iSUT state.

My isut is set to autodeploy. AMS state is ok

ILO amplifier pack did the job, now this plugin for vcenter and oneview is a big mess! Nothing works

[AR-J]

In a similar boat, trying to migrate from amplifier now that it no longer works

I'm seeing the following in the OV4VC log bundle, my check/remediation attempts return pre-check FAILED

Traceback (most recent call last):
  File "/opt/ov4vc/ovm/src/vlcm/actions/cluster_vlcm_precheck_action.py", line 46, in get_isut_ams_status_of_host
    ilo_redfish_client = OVMiLORedfishClient(https:// + ilo_ip, ilo_sessionKey)
  File "/opt/ov4vc/ovm/src/ilo_wrapper/ilo_redfish_client.py", line 25, in __init__
    self.typepath.getgen(url=host_ip, logger=log)
  File "/opt/ov4vc/ovm/ovmenv/lib64/python3.8/site-packages/redfish/ris/tpdefs.py", line 79, in getgen
    self.ilogen = rootresp["Oem"]["Hpe"]["Manager"][0]["ManagerType"]
KeyError: 'ManagerType'
2023-09-11 12:35:42,765 2988 139658475398912 ERROR vlcm.actions.cluster_vlcm_precheck_action get_isut_ams_status_of_host Error occurred in getting isut and ams info for hosts and error is 'ManagerType'
Traceback (most recent call last):
  File "/opt/ov4vc/ovm/ovmenv/lib64/python3.8/site-packages/redfish/ris/tpdefs.py", line 77, in getgen
    self.ilogen = rootresp["Oem"]["Hp"]["Manager"][0]["ManagerType"]
KeyError: 'Hp'

vLCM Pre-Check iSUT State is shows a red icon against all my hosts with an unknown status, attempts to configure it with the UI end up with the tasks failing at 80% with those failures lining up with the above log entires.

Using: OV 8.5, OV4VC 11.4, VC 7.0.3n, ESXi 7.0.3m, SPP Gen10 09.2023, DL380 G10

[vmuniverse1]

I have exactly the same thing. Stuck at 80% and then it gives an error

Verzonden vanaf Outlook voor Android<>

[vmuniverse1]

Anyone knows what's causing this?

[RenjuCR]

Looks like a configuration issue to us.

Please raise a support ticket with HPE (Hewlett Packard Enterprise) so that technical support team can investigate the issue, provide configuration info if needed, review the logs, and take appropriate actions to resolve the problem.

[AR-J]

I tried that and got the run around of being told to do steps id already stated id done, several times. I've wasted a significant amount of time going over redoing those steps trying to make this thing work already, I didnt need support to tell me to set AutoDeploy if I've done it 10 times already, I asked to get someone on a call so they could confirm rather than wasting more of my time.

I have since solved this myself, the issue is that regardless of the ilo already being setup in oneview the pre-check attempts a getgen call in the python script, it does this without providing credentials and fails. The only thing that code does is check if its version 5 or higher but that trips up the entire process.

This would have been easier to solve if there was any useful logging implemented at all, if it were documented somewhere what settings in the ilo might be required, and if it were documented what the 'Anonymous Data' flag in the ilo actually affects (the View XML link next to this tickbox does not cover it)

It shouldnt take me as an engineer having to go and find the python code to what that function was actually doing, testing the redfish API and seeing the missing fields

[RenjuCR]

Happy to hear that your issue is resolved.

Please create a ticket describing the precise issue you had with the documentation, so that we can follow it up and track it towards a closure.

[AR-J]

I have my own tasks to get on with, I dont work for HPE support

[bkpht]

Was this somewhere solved yet?

I have the same issue

[vmuniverse1]

Enable Anonymous Data in ILO. That should fix it

[bkpht]

Checked it and it is already enabled, so other tips are welcome

[vmuniverse1]

Check your ISUT settings. Make sure its set to Autodeploy. Use the latest ISUT. Reset your ILO and enable the Anonymous checkbox.

[RenjuCR]

Server profiles are not mandatory.

We hope you have made sure that ILO in production mode.

Also FIPS/CNSA and other security modes are not supported in OV4VC.

[pirx]

Don't bother too much with OV and OV4VC. Even if you get it in a state where it works... it doesn't. OV4VC 11.5 was the first version that worked for me in any way, before 'monitor mode' was not working at all or it was using force install mode. But even with 11.6 it failes to remediate half of a cluster. Updates are done succesful in ILO but the compliance check failes. and all next updates stop. Manual recheck results in a compliant host and I can restart remediation. There are so many quirks that it's just not usable in production.

Check if all host are really set to AutoDeploy, see my post at the beginning. I changed it on cli but it did not rellay change and I had to first set a different mode and then switch back to AutoDeploy. Check if communication between ILO, OV, OV4VC, vCenter is somehow blocked.

HPE OneView Ports - Hewlett Packard Enterprise Community

[bkpht]

thnx for the replies.

all of the above is already set and tested. I even installed all latest versions, now i have OV 8.8, ov4vc 11.6, iSUT 5.0

first pre-check gives me this

when i remove the hosts from vcenter and add it again, i get his 

while all is set on Autodeploy the iSUT state is not correct, start the configuration from here will end up in hanging at 80% and failes.

when i leave this page and come back to it i get my first error again.

some servers are working so i guess firewall rules etc. are setup correct. Servers are all installed the same way with autodeploy, hostprofile and HPe image.

maybe i have to give up pirx mentionned

[vmuniverse1]

I think hoe oneview 8.9 and isn't 5.5 are the latest. It's important to get the latest versions because hpe fixes problems in new releases.

[bkpht]

I will checkout the OV version but for iSUT 5.0.0 is the latest

[pirx]

Don't update to OV 8.90. It's not working with vLCM for me and others, check the post here in community about that.

What ILO version do you use, maybe it's worth updating it to the lastes first. Can you post the isut state on ESXi?

# sut /status
System Update Manager...................: HPE OneView
Update Manager URL......................: https://10.11.5.54
Deploy Type.............................: FirmwareOnly
Task Status.............................: Bios:N/A; LocalStorage:N/A; Deploy:Activated
Staging Directory.......................: /tmp/stagingdirectory
Baseline URI............................: None
Baseline Version........................: Waiting for staging
Mode of Operation.......................: autodeploy
Polling Interval In Minutes.............: 5
Force Deploy............................: No
Optional Components.....................: None
EnableiLOQueuedUpdates..................: true
User Action Needed......................: None. Activation is complete and node is updated with the current baseline
iLO Security Mode.......................: Production
iLO CAC Smartcard Authentication........: Off
iLO CAC Strict Mode.....................: Off

# sut /version
Copyright (C) 2019 Hewlett Packard Enterprise Development LP
Integrated Smart Update Tools version 5.0.0.

[bkpht]

will try with the latest ilo version as for now it is still 2.99 on my test server

isut settings

[vmuniverse1]

You need to go to ILO 3.0.X I had contact with HPE and they confirmed there was a problem with ILO and vlcm in earlier release.

Verzonden vanaf Outlook voor Android<>