Server Management - Remote Server Management

Trouble with iLO upgrade using powershell

 
uzimmermann
Advisor

Trouble with iLO upgrade using powershell

So this has worked many times before but now I am getting problems.

Linux Powershell 7.1.1 with HPEiLOCmdlets 3.0.0.1:

> $conn2 = $ilos2 | Connect-HPEiLO -Credential $ilocred -DisableCertificateAuthentication

> Update-HPEiLOFirmware -Location "/home/XXX/iLO/firmware/iLO5/ilo5_233.bin" -Connection $conn2

Confirm
Are you sure you want to perform this action?
Performing the operation "Update-HPEiLOFirmware" on target "10.237.75.22".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): a
WARNING: Update is in progress, this might take several minutes.
Update-HPEiLOFirmware: [10.237.75.22]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.23]: The terminal URI could not be found for odataType UpdateService with traversal path ['Oem'].['Hpe'].['Actions'].['#HpeiLOUpdateServiceExt.AddFromUri'].['target']
Update-HPEiLOFirmware: [10.237.75.72]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.73]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.74]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.75]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.76]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.77]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.172]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.173]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.174]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.175]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.231]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.232]: Object reference not set to an instance of an object.
Update-HPEiLOFirmware: [10.237.75.233]: Object reference not set to an instance of an object.

Same with Powershell on Windows:

Powershell version 5.1 build 14383 Revsion 3866 with HPEiLOCmdlets 3.0.0.1

PS C:\Users\XXX > $conn2 = $ilos2 | Connect-HPEiLO -Credential $ilocred -DisableCertificateAuthentication

PS C:\Users\XXX> Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin -Connection $conn2

Confirm
Are you sure you want to perform this action?
Performing the operation "Update-HPEiLOFirmware" on target "10.237.75.22".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): a
WARNING: Update is in progress, this might take several minutes.
Update-HPEiLOFirmware : [10.237.75.22]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.23]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.72]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.73]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.74]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.75]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.76]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.77]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.172]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.173]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.174]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.175]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.231]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.232]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

Update-HPEiLOFirmware : [10.237.75.233]: Object reference not set to an instance of an object.
At line:1 char:1
+ Update-HPEiLOFirmware -Location C:\Users\XXX\Downloads\ilo5_233.bin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Update-HPEiLOFirmware], Exception
+ FullyQualifiedErrorId : HPE.iLO.Cmdlet.UpdateHPEiLOFirmware

This used to work just fine, I have done at least 10 rounds of upgrades this way. I am stumped.

Any ideas?

 

9 REPLIES 9
PowerShellLover
Occasional Visitor

Re: Trouble with iLO upgrade using powershell

We have the same problem with the new ilo5_233.bin firmware image. It happens only with this file. The Update-HPEiLOFirmware works with the new ilo4_277.bin without problems. There seems to be a problem in the filehandling (buffer underun?). Nothing we can do about it without a fix by hpe.

Tested with HPEiLOCmdlets 3.0.0.1 and HPEiLOCmdlets 2.x

For reference and google purposes the german error messages:
FullyQualifiedErrorId >> HPE.iLO.Cmdlet.UpdateHPEiLOFirmware
Exception.Message >> Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.

PS: This is NOT a "FullyQualifiedErrorId" dear HPE programmers...

uzimmermann
Advisor

Re: Trouble with iLO upgrade using powershell

At least under Linux I have the same problem with the iLO 2.77 file, I don't think I tried that one under Windows.

 

AmRa
HPE Pro

Re: Trouble with iLO upgrade using powershell

We have seen this issue with iLO 5 2.33 and Windows PowerShell iLO Cmdlets Version:3.0.0.1,  Same issue has been highlighted to HPE Engineering team. 

 

HPE backend team is working on this issue and this may fix in new HPE Windows PowerShell iLO Cmdlets release.

I am an HPE Employee

Accept or Kudo
Questions
Visitor

Re: Trouble with iLO upgrade using powershell

I hope the "may fix" for this is an understatement.

When is an ETA for an actual fix for it?

/Henke

Steam_E
Occasional Visitor

Re: Trouble with iLO upgrade using powershell

Dears

 

We have also this bug this some months. At this moment we are not more able to maintain all of our servers up-to-date.

Do you have in meantime a corrective or a workaround? Thanks

uzimmermann
Advisor

Re: Trouble with iLO upgrade using powershell

HPEiLOCmdlets 3.1.0.0 which I just installed a couple days ago made this work again.

 

Luke Hsieh
Occasional Advisor

Re: Trouble with iLO upgrade using powershell

While the upgrade feature was fixed with HPEilocmdlet 3.1.0.0 and ILO5 2.3x, I found the cmdlet to configure Directory Group is still broken.  I complained about it in March and got response sayind it's being worked on and they're aiming to release update in April.  Not too many days in April left though...

PowerShellLover
Occasional Visitor

Re: Trouble with iLO upgrade using powershell

I can confirm this. With HPEiLOCmdlets 3.1.0.0 the Update-HPEiLOFirmware commandlet works again for binary iLO-Update files.

But it seems there was a change in the behaviour of Update-HPEiLOFirmware when sending an iLO-Update file. It returns as soon as the file is completely uploaded. In the versions before it returned after the update was flashed. Now I seek a way to check if the flashing of the update is still running (like the the box "Flashing Firmware x%" shown when logged into the ilo via browser). But I cannot find a cmdlet showing this info. The only thing I found was checking the eventlog for firmware upgrade events (via (Get-HPEiLOEventLog).EventLog | where {$_.Message -like '*Firmware*Upgrade*'} ). But this is not perfect in any way...

Any ideas?

PowerShellLover
Occasional Visitor

Re: Trouble with iLO upgrade using powershell

I have looked at this problem a little bit deeper.

iLO Generation 4 uploads and flashes until Update-HPEiLOFirmware returns an empty object.

iLO Generation 5 only uploads until Update-HPEiLOFirmware returns an StatusInfo object with message ResetInProgress.

More details on ResetInProgress message:

$output = Get-HPEiLORedfishMessageInfo -Connection $hpe_ilo_con

$output.ResetInProgress


Description : A management processor reset is in progress.
Message : A management processor reset is in progress.
NumberOfArgs : 0
ParamTypes : {}
Resolution : Wait for management processor reset to complete, and then retry the operation.
Severity : Warning
Oem :

 Which is strange because when the warning is returned by Update-HPEiLOFirmware the ilo starts to verify the uploaded data and then flashes the bin file. So there is no reason for the function to return an error...

tested with: HPEiLOCmdlets 3.1.0.0, iLO4 2.77/2.78, iLO5 2.41/2.44

Has anybody else seen this?