DL20 Gen10 Fan3 speed control is terrible.

[BunnyPon]

What we have here is a DL20 gen10 running debian 12 + Influxdb/telegraf/grafana.

Some of this data is from redfish (which sucks because SNMP was easier to use but gives different answers for no valid reason except to make you use redfish.) and some from "smartctl" The important things here are, 

The drive temperatures, and the bottom graph lilac line (Average drive temperature) and the bold red line doing comb impressions.

Note that these intel SSDs do have the right SMART answers to stop the raid controller losing its marble and going to take-off thrust. (As discussed elsewhere.)

The lilac line and the red line show an obvious correlation and dismal speed control going from 10% (acceptable) to 16% (annoying) But what is even more annoying is the continuous oscillation. (20% you don't want to be in the same room).

Surely we can do better than this naive proportional control of Fan3? I wouldn't recommend one of these for a quiet office at all. Grafana on DL20 Gen10

[rabindra11sharm]

Dear  BunnyPon

Please share your servers IML(Integrated Manegment Log), and firmware version. Also Please inform which version of debian is used on it.

[Cali]

The smaller the fans, the faster they have to rotate and the louder they are.
Yes, these little 1U fans have an annoying high-pitched noise.

That's why a 1U server is a bad choice if you want it to be quiet.

A 2U server is much quieter and for noise-sensitive rooms I always use a 3U server (ML), the difference in noise is enormous.

Cali

[BunnyPon]

I know, and I completely agree.

I also have a ML30 Gen10 and this strange thing with some sort of OEM crap that I can't get rid of that is an ML30 Gen10 which I keep for parts because the "hitachi profile" I can't get rid of stops me buying an ILO Advanced license because it WILL NOT WORK.

If HP made a 2U the same size as the DL20, it would be excellent, AND you could get get 2 of them in a rack.

The only reason for the DL20 is the TM200 only has one fan and it is way noisier (but oh so CUTE!)

[BunnyPon]

Here's the latest data from the ILO + what telegraf is seeing. 

As you can see there is no happy medium between 10% and 16%.

What sensor is Fan3 using exactly? I can add 04-HD Max if needed, but fundamentally this is naive proportional control with possibly some hysteresis. It could be a lot better.

Temperatures24 hours, look at the comb effectLast 3 hours showing the comb in more detail.

OS: Debian 12.6,  

Package: amsd
Version: 3.3.0-1772.24-debian12  (( I do like how they call it "agentless" yet you have to install an agent...))

Firmware:

1TB 6G SATA HDD	HPGC	Slot=0:Port=1I:Box=2:Bay=3	
1TB 6G SATA HDD	HPGC	Slot=0:Port=1I:Box=2:Bay=4	
80GB 6G SATA SSD	D2010310	Slot=0:Port=1I:Box=2:Bay=1	
80GB 6G SATA SSD	D2010310	Slot=0:Port=1I:Box=2:Bay=2	
Embedded SATA Controller #1	N/A	Embedded SATA	
Embedded Video Controller	2.5	Embedded Device	
HPE Ethernet 1Gb 2-port 332i Adapter - NIC	20.26.42	Embedded LOM	
HPE Smart Array E208i-a SR Gen10	6.52	Embedded RAID	
iLO 5	3.06 Jul 10 2024	System Board	
Intelligent Platform Abstraction Data	16.1.0 Build 21	System Board	
Intelligent Provisioning	3.88.6	System Board	
Power Management Controller Firmware	1.1.4	System Board	
Power Management Controller FW Bootloader	1.1	System Board	
Power Supply Firmware	1.00	Bay 1	
Power Supply Firmware	1.00	Bay 2	
Redundant System ROM	U43 v3.00 (02/01/2024)	System Board	
Server Platform Services (SPS) Firmware	5.1.4.913	System Board	
System Programmable Logic Device	0x0C	System Board	
System ROM	U43 v3.10 (05/16/2024)	System Board	

As you can see, I haven't updated it to 3.07 and the latest bios. The 4 drives you see are just JBOD off the controller.

IML:

Regarding the IML, I see nothing interesting in there since installing the bios/ilo updates on July 16th. Only the usual NIC up/down and adding the HDDs above. Before that there was only playing with drives until I found SSDs that worked without it going into TURBINE mode. 

"ID","Severity","Class","Description","Last Update","Count","Category",
"474","Repaired","Network","One of the NIC connectivity status changed to OK","07/21/2024 16:08:24","1","Hardware",
"473","Critical","Network"," All links are down in adapter HPE Ethernet 1Gb 2-port 332i Adapter - NIC in slot 0","07/21/2024 16:07:26","1","Hardware",
"472","Critical","Network","One of the NIC connectivity status changed to Link Failure","07/21/2024 16:07:24","1","Hardware",
"471","Informational","System Revision","Firmware flashed (Online ROM Flash Firmware Package - HPE Integrated Lights-Out 5 3.06)","07/17/2024 03:20:17","1","Administration",
"470","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 17:20:59","1","Hardware",
"469","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 05:57:30","1","Hardware",
"468","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 05:57:13","1","Hardware",
"467","Informational","Maintenance","Maintenance note: Starting F10 GUI","07/16/2024 05:42:14","1","Maintenance",
"466","Informational","Maintenance","Maintenance note: Running scripting interface...","07/16/2024 05:41:59","1","Maintenance",
"465","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 05:41:38","1","Hardware",
"464","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 05:37:32","1","Hardware",
"463","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 1 is Inserted","07/16/2024 05:30:04","1","Hardware",
"462","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 2 is Inserted","07/16/2024 05:32:00","1","Hardware",
"461","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 1 is Inserted","07/16/2024 05:25:16","1","Hardware",
"460","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 05:14:04","1","Hardware",
"459","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 05:13:47","1","Hardware",
"458","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 1 is Inserted","07/16/2024 05:11:25","1","Hardware",
"457","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 2 is Inserted","07/16/2024 05:11:25","1","Hardware",
"456","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 1 is Inserted","07/16/2024 05:00:19","1","Hardware",
"455","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 2 is Inserted","07/16/2024 04:47:07","1","Hardware",
"454","Repaired","Drive Array","Slot 0 Smart Array - SSD Wear Status Level Ok: Port 1I Box 2 Bay 2","07/16/2024 04:42:12","1","Hardware",
"453","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 2 is Inserted","07/16/2024 04:42:00","1","Hardware",
"452","Repaired","Drive Array","Slot=0 Storage - Drive at Port 1I Box 2 Bay 2 is Inserted","07/16/2024 04:33:15","1","Hardware",
"451","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 04:26:20","1","Hardware",
"450","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 04:22:38","1","Hardware",
"449","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 04:18:22","1","Hardware",
"448","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 04:18:04","1","Hardware",
"447","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/16/2024 01:22:34","1","Hardware",
"446","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 15:49:34","1","Hardware",
"445","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 13:50:02","1","Hardware",
"444","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 13:49:12","1","Hardware",
"443","Informational","Maintenance","Maintenance note: Starting F10 GUI","07/15/2024 13:45:06","1","Maintenance",
"442","Informational","Maintenance","Maintenance note: Running scripting interface...","07/15/2024 13:44:50","1","Maintenance",
"441","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 13:44:19","1","Hardware",
"440","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 13:44:02","1","Hardware",
"439","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 11:53:05","1","Hardware",
"438","Informational","Maintenance","Maintenance note: Starting F10 GUI","07/15/2024 11:38:33","1","Maintenance",
"437","Informational","Maintenance","Maintenance note: Running scripting interface...","07/15/2024 11:38:18","1","Maintenance",
"436","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 11:37:41","1","Hardware",
"435","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","07/15/2024 11:33:38","1","Hardware",
"434","Informational","System Revision","Firmware flashed (System BIOS - U43 v3.10 (05/16/2024))","07/15/2024 11:27:16","1","Administration",
"433","Repaired","Network","One of the NIC connectivity status changed to OK","07/15/2024 11:23:41","1","Hardware",
"432","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","[Not Set]","1","Hardware",
"431","Informational","System Revision","Firmware flashed (Online ROM Flash Firmware Package - HPE Integrated Lights-Out 5 3.05)","[Not Set]","1","Administration",
"430","Repaired","Network","HPE Ethernet 1Gb 2-port 332i Adapter - NIC Connectivity status changed to OK for adapter in slot 0, port 1","[Not Set]","1","Hardware",

[rabindra11sharm]

Dear BunnyPon 

Please update latest iLO 5 firmware 3.7 also please install Agentless Management Service (AMSD). it will reduce the noise and you will be able to work comfortably in your office.

 

[BunnyPon]

Incidentally, after updating to 3.07 and bios 3.40 here is the graph from cold to what looks like steady state 1'c cooler.

was turned off for about 20 minutes to tidy up the spaghettii!

Now, whether this is the ILO being just a little smarter or not I am not sure. Instead of 10~16% at average 39'c, we have 13~14% at 38'c. with spurious drops to 12% but never 10%.

04 HD Max vs FAN3

It would really help to know what drives the fan temperature. If it is 04-HD Max 

[rabindra11sharm]

Dear BunnyPon

HPE Agentless Management offers a consistent Graphic User Interface (GUI) for configuration regardless of the host OS. An intelligent asynchronous device, iLO, on the physical server providing a single network target to monitor and manage the device. Please trying to install the AMS service on your host OS. it might be resolved the FAN noise issue. earlier we have seen in windows and RHEL OS after install this service FAN issue resolved. 

Please download it from HPE repository, and try it.

https://downloads.linux.hpe.com/SDR/repo/mcp/Debian/pool/non-free/

[Rakesh0404]

Hi, 

Let us know if you were able to resolve the issue.

If you have no further query, and you are satisfied with the answer then kindly mark the topic as Solved so that it is helpful for all community members.

[BunnyPon]

That reads like the robots are in control. This does not answer the question of who/what drives FAN3.

AMS is indeed installed. If you looked, I even mentioned the version. Calling it "Agentless management" when you have to install an agent though, is at best a misnomer, at worst written by the marketing crimethink department.

No one needs a GUI on a server, they live far away and very often the user will never see the physical machine.

[Rakesh0404]

Dear BunnyPon, As per the suggestion shared previously by the Eng, the AMS install should have resolved the issue. 
However, in this scenario the OS is Debian, and the AMS is installed , still the fan noise is high. 
We request you to log a case on HPE Support Center portal for further resolution using the link: https://support.hpe.com/hpesc/public/home/

[BunnyPon]

As per the previous reply.

It is installed. Since the inner workings are opaque unless we go looking on github for an ancient version. We still have no idea why FAN3 is going zoom, why it has such crap proportional control with no hysteresis or what. While it is a lovely server, I will have to wait until gen11 become affordable.

Please explain exactly which sensors control FAN3.

Updating to 3.07 did make a definite improvement, but it could also be that it warmer today.

The change in fan three from 16% to 19% is enough that one really wants a 2U version of the DL20 because other than noise, they are really capable toys.

Package: amsd
Version: 3.3.0-1772.24-debian12
Priority: optional
Section: admin
Maintainer: mark rusk <mark.rusk@hpe.com>
Installed-Size: 7,116 kB
Depends: libc6 (>= 2.34), libgcc-s1 (>= 3.0), libjson-c5 (>= 0.15), libstdc++6 (>= 11), libsystemd0, lsof
Homepage: http://linux.hpe.com
Download-Size: 1,124 kB
APT-Manual-Installed: yes
APT-Sources: http://downloads.linux.hpe.com/SDR/repo/mcp bookworm/current/non-free amd64 Packages
Description: Agentless Management Service
 Provides RFC 1741 AgentX sub-agents and AgentX proxies, which allow the iLO5
 Agentless Management facility (SNMP) access to OS information, and OS SNMP
 agentx access to iLO SNMP sub-agents.

[BunnyPon]

Since this DL20 is out of support, it threatens me with "this might cost money" which considering how much I paid for it is a major disincentive for what is effectively a toy.

And as previously mentioned but completely ignored by the robot, AMSD is installed and if it has a graphical interface like the robot said, then that is news to everyone. Please have your robot sent to the re-education camp.

With 3.07, fan noise could be described as "better" but still too high and varies too rapidly like it is a naive proportional control. I look forward to gen11 things (or even gen10plus) becoming affordable on the second-hand market and I will be retiring this as soon as I can find something cuter^Wquieter.

There are no clues in the release notes if 3.0.8 will help.