ESXi host memory health warning - iLo 5 v2.30

[PatrickLong]

There appears to be an issue with new  Gen10 iLo5 firmware 2.30 and ESXi 6.7 memory health warnings.  After upgrading my Synergy SY480 and SY660 blades to the latest 2.30 release of iLo5 firmware, within a short while (not immediately) all of these hosts are showing a red bang error for Host Memory Status.  If I Reset to Green, the error clears for a short while, but then eventually returns..  On important item I've noticed is that when the host is in the Error state under iLo5 2.30 , under Monitor > Hardware Health I see new sensors for memory with names like "Memory Device 1 Mem_Stat_C01S02" in  Alert status with Reading of either 128 or 192 as shown:

However, when I back-rev the iLo to previous version 2.18 and reboot the host, the errors are gone as are the newer memory sensor readings:

So it appears that iLo5 2.30 introduced NEW sensor readings for memory that ESXi is not handling properly - and I am using the very latest build release VMware ESXi 6.7.0 build-16713306 release date 8-20-2020 and all latest firmware/drivers/HPE utils esxi6.7uX-mgmt-bundle-3.5.0-12  and esxi6.7-util-bundle-3.5.5-3, as well as latest BIOS I42 v2.36 7/16/2020, Innovation Engine 0.2.2.0, and Server Platform Services 4.1.4.381.1. As well as latest OneView Composer 5.40.00-0423953\.  This seems very similar to this old sensor-reading chestnut from 2018/19: with ESXi and iLo5 v1.30  https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00053955en_us

Anyone else experiencing the same issue?  I don't have any non-Synergy Gen10 hosts to test this on to see if the issue is consistent across all Gen10 / iLo5 systems and ESXi but my guess would be that it is.  And yes, I'm well aware that the latest Synergy Custom SPP 2020.07.01 (or " Service Pack for ProLiant, SY-2020.07.1" as it's apparently being called now - but don't try running it on a ProLiant server, it won't work) includes iLo5 v2.18, NOT 2.30 - however the 2.30 fixlist specifically includes a fix for Synergy:

"Synergy compute modules do not power up after One-button secure erase completes."

which makes it a bit of a hard sell to say "Don't install 2.30 on Synergy".

[netscaper71]

Hello, we have same issue exactly same. 6.7 fresh updates from friday and ilo5 ver2.30 on 5 servers
ProLiant DL360 Gen10
Memory Device 3 Mem_Stat_C01S04

[ArminasU]

Hi, I have same issue on all hosts ProLiant DL360 Gen10. Updated ESXi to version 6.7 16713306 and ILO5 to v2.30.

ESXi hardwre health Alerts:

Memory Device 10 Mem_Stat_C01S11

Memory Device 11 Mem_Stat_C01S12

Memory Device 18 Mem_Stat_C02S03

When we can expect new ILO5 update version with fix or i have to downgrade to ILO5 version 2.18?

[Mountain]

Same to us, DL380Gen10 - put it back to 2.19   Was a advise from support and they are in contact with vmware

[David Kobler]

same here

[fojtp]

Gen10 hosts affected (BL460c, DL380) with ESXi versions (6.5 - tested latest build 16576891, 6.7 - build 16316930).
ESXi 7.0 (latest build 16324942) seems OK because Sensor Status is Uknown (for Memory Device)

[MK_J]

It seems iLo5 FW 2.30 exposes some additional sensors and alerts are triggered due to this. 

To avoid getting these memory alerts, you may remain at N-1 FW version.

Please open a support case with HPE Synergy so that you would be updated with develpoment on this issue.

Best Regards,

Milan

[AmritRanjan]

This issue is already under investigation at HPE.

If you feel this was helpful please click the KUDOS! thumb below!   

Regards,

[SanjeevGoyal]

Hello,

I hope all is well.

I would suggest you to have a proper case be logged with HPE, and share the appropriate logs for further analysis.

If you feel this was helpful please click the KUDOS! thumb below!   

Regards,

[David Kobler]

Is there an oficial HPE document/advisory for the issue?