HSM plugin with iLO Amplifier Pack issue

[Safi7]

Hello,

So we deployed iLO Amplifier Pack and integrated with vCenter successfully, imported Cert and all looks good.

Then we started to map each host server to iLO by adding credentials in HSM plugin configure page in each host.

All hosts we're connected with Amplifier successfully except for 2 hosts which always give out this error 500 when trying to add iLO host credentials:

And I can;t find any logs for this.

I tried to add the server to the iLO Amplifier directly via its webgui and it worked, but still can't connect the host in vCenter to its corresponding iLO.

Note: I already attempted to uninstall "amsdComponent" and reinstall latest version with no luck.

Please advise how this can be fixed..

[support_s]

System recommended content:

1. HPE Integrated Lights Out (iLO 4) - Troubleshooting Firmware Update Issues

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Safi7]

iLO is updated and seem to work normally, I tried downgrading the iLO version and upgrade to latest 2.78 iLO 5 but same issue.

In iLo logs I can see in sessions that when I attempt to add the ilO in HSM a session of type "REST host" is created, but keep getting this unclear error "some error occured..."

Can you maybe advise from where I can find more detailed logs for this operation?

In current logs I Can only see that host keeps logging in and out

[Safi7 · Accepted solution]

UPDATE:

So for the frustrated guys that will follow, I managaed to solve the problem by downgrading the entire server firmware + iLO firmware, then I was able to integrate the host with HSM successfully.
 

[Sunitha_Mod]

Hello @Safi7, 

Perfect! 

We are glad to know you were able to solve the problem and we appreciate you for keeping us updated. 

[pbreitwieserAH]

@Safi7 

Currently running into this issue with vcenter trying to connect to iLo5 version 2.72.  What version did you end up downgrading to?

[Safi7]

I honestly don't recall which version it was, but in your iLO -> Software & Firmware -> Install sets
You can find the system recovery set, you can use it to restore older firmware/iLO, it should work.

Note: As per official HPE response iLO amplifier support will be dropped in the very near future, as they are shifting all focus to OneView.

[kL4]

@pbreitwieserAH Did you manage to solve this issue aswell ?

I tried everything, downgrading/upgrading server firmware and iLO firmware, trying to reinstall from the recovery set, even tried a full factory reset. It just won't work.

I'm lost for ideas right now...

[pbreitwieserAH]

@kL4 , I have not.  I haven't wanted to downgrade the firmware and move off a supported vLCM combo.  I will be updating to the newest valid combo propbably within the next month since this issue isn't mission critical.

That being said, I did find some information in the logs for the HPE HSM plugin located on my iLO Amplifier that points shows the ipv4address field being returned blank at somepoint during the connection process.  The username that I provide does successfully authenticate with the iLO and Rest API, but fails at some point afterward (See below).  

2023-04-18 10:33:08,692 - vum_sum_ti.hpe_hosts_api_controller - INFO - Reading the task information for 1ceb4716-0272-4b05-bc30-53e53bdcebb3
2023-04-18 10:33:08,801 - vum_sum_ti.ilo_ris_client - INFO - iLO REST login is successful for gs-ilo-esxi02.ad.scgsah.org
2023-04-18 10:33:08,939 - vum_sum_ti.vApiClient - INFO - Login Successful
2023-04-18 10:33:08,939 - vum_sum_ti.vApiClient - INFO - Getting a session clone ticket
2023-04-18 10:33:08,947 - vum_sum_ti.vApiClient - INFO - Got a session clone ticket succesfully.
2023-04-18 10:33:08,951 - vum_sum_ti.vApiClient - INFO - Logout successful
2023-04-18 10:33:08,952 - vum_sum_ti.authenticator - INFO - Authentication Successful for 'hpe_internal_get_task_info'
2023-04-18 10:33:08,969 - vum_sum_ti.hpe_hosts_api_controller - INFO - Reading the task information for 1ceb4716-0272-4b05-bc30-53e53bdcebb3
2023-04-18 10:33:08,985 - vum_sum_ti.ilo_ris_client - INFO - iLO REST logout is successful for gs-ilo-esxi02.ad.scgsah.org
2023-04-18 10:33:08,986 - vum_sum_ti.hpe_hosts_api_controller - ERROR - 'IPv4Addresses'
2023-04-18 10:33:08,986 - vum_sum_ti.hpe_hosts_api_controller - INFO - Task Comeplete.
2023-04-18 10:33:08,986 - vum_sum_ti.hpe_hosts_api_controller - INFO - Task output: {'operationStatusCode': 500, 'messages': ['Some error occurred during the task'], 'id': '1ceb4716-0272-4b05-bc30-53e53bdcebb3', 'description': 'Set ilo for the host', 'action': 'SET_HOST_ILO', 'startTime': '2023-04-18 14:33:08.079617', 'clusterId': 'domain-c21', 'hosts': ['host-10'], 'status': 'FAILED', 'hostToIloMap': {'host-10': {'iloIpAddress': 'gs-ilo-esxi02.ad.scgsah.org', 'osIpAddress': '10.0.1.182', 'status': None}}}
2023-04-18 10:33:09,000 - celery.app.trace - INFO - Task swagger_server.controllers.hpe_internal_hosts_api_controller.hpe_internal_set_host_ilo_task[0df82644-8ebb-4447-b322-e5676a824ab2] succeeded in 0.8469129100000146s: None

The log file (vum_sum_ti.log) can be found in the iLo Diagnositcs Log pack in the folder logs\opt\wfm_plugins\hpehsm\logs\vlcm-hsm if you want to check yours.

Whether or not this is an iLO amplifier issue or iLO is yet to be seen.  I'll be submitting a ticket with this information in the coming days.

[Safi7]

2023-04-18 10:33:08,986 - vum_sum_ti.hpe_hosts_api_controller - ERROR - 'IPv4Addresses'

For that error, it could be a DNS issue, I would suggest to check that you have correct (A/PTR) records of (VC/ESXis/HSM), and that resolution and reverse resolution works from all these components between each other.

[pbreitwieserAH]

Thanks.  Checked all and verified both resolution and reverse resolutions work on VC/ESXis/HSM.  Guess I'll send in a ticket for the time being and see if any more info can be gathered.

[kL4]

I did the same check as you did within the vum_sum_ti.log file, and i can find the same error as you did.

I also checked bot resolution and reverse resolution, so far so good.

This is weird tho, i don't get how a simple iLO firmware update could prevent me from the registration (right now i have 4/6 servers that made it through the registration, and they're all alike). The only difference was an ilo firmware update on the first 2 servers.

[CBVista]

Also affected by this. All servers are identical FW/Baseline, but only 1 of the 4 servers added successfully.