DL 385 Gen11 High fan speed

[JensA]

Hi folks.
Has anyone recentli had any issues with a DL385 GEN11, and high fan speeds?
When starting the fan speeds go up as normal, i would say about 40%-50% (Is not displayed in iLO when booting).
And when starting the OS, the fan speed goes up verry high, and then it is random if the fan speed goes down again, or it is stuck at high speed for example 68%. All fans will stay at the stuck speed, unless i set it hither in the iLO. When i set it to "0%" it will go back to 68%.

The stuck speed changes pr. reboot, sometimes it is 61%, sometimes is it 72%.

Looking in device manager, the NVMe drives are missing when the fan speeds are high. But all drives and raids are working.
When all the NVMe drives are displayed, the fan speeds are normal (11idle 23% full CPU load).

Server info, and what i have done:
- The first things i did, was update iLO, BIOS and other firmware/driver with SUM. And the issue is still there.

- I was told the 1G NIC from Intel an advicory about high fan speeds, so i removed it. Same issues.
  https://support.hpe.com/hpesc/public/docDisplay?docId=a00126095en_us&docLocale=en_US

- I have a brand new server, AMD EPYC 9174F, 750GB memory and 8NVMe drives.
I'm running VMWare 8 update 2, HPE branded.

- I tried VMWare 7x and 8x, and fress installs, HPE branded ISO's.


I also opened a case with HPE 3 weeks ago. And it is escalated to an engeneer, but the responses are verry slow, and I'm trying to figureout, what is happening.

Need some info on other cases on this, maby sulitions or input.

Jens.

[rabindra11sharm]

Dear JensA

As per my suggesion, update all firmwares by latest SPP for gen 11  also you can use your 1G NIC card. it is working and design for it(As per shared Advisary). you may set your server profile to RBSU/BIOS menu for better performance as per your reqairments System Utility Guide for Gen11 Server

[JensA]

Hi again.
The SPP is baseline and above, also firmware updated with the newest bios today 1.62.
And i have tried diffrent profiles, same issue.

/Jens.

[MGoorman]

Looking in device manager, the NVMe drives are missing when the fan speeds are high. But all drives and raids are working.
When all the NVMe drives are displayed, the fan speeds are normal (11idle 23% full CPU load).

This seems to be your issue. 
It looks like there is some communication failure.
When you don't "see" your NVMe drives (I think you mean that the drives are not displayed in the iLO?), the iLO knows there is a device but can't communicate with it, so to be sure the fan speed is high to cool the devices.
When your devices are shown, the iLO can do a temp readout and will adjust the fanspeed.

Now I'm not a VMWare specialist, but on Windows this should be the Agentless management service that takes care of this communication.
I don't know what the equivalent is on VMWare.

[JensA]

I agree with you.

The devices are not displayed in the invantory in iLO, and when i do a iLO reset, it works again.

AMS i think is also used in VMware, and the AMS services also has a green checkmark.
I have shared this information with HPE for a solution, but i still don't have one.

/Jens.

[MGoorman]

Do you know what version of the AMS Bundle you are using?
Maybe you could update it:
https://support.hpe.com/connect/s/softwaredetails?collectionId=MTX-3acaa73d0fea4c6c&softwareId=MTX_aa4a62b4ce464e04b8538cddac&tab=revisionHistory

[TVVJ]

Hello,

You may try installing the driver for the installed storage controller and see if it helps.

Regards,

[JensA]

I think the driver should be there with the HPE branded image for VMWare.

Also i have run SPP.

[JensA]

Hi.
The one I'm using 701.11.5.0 , so there is one there is a bit newer.
I'm looking in to it.

[JensA]

@rabindra11sharm I did this and rebooted some times, and the issue is still there.

/Jens.

[rabindra11sharm]

Hi Jens A

I am very sorry hear that you still facing issue on your server. 

Can you please help me to understand your server configuration. Please check your server BOM details to check configuration. or you can check your server hardwares spare part no. I think there is some mismatch configuration.  

Earlier your shared data I can see you are using a Epic 9174F PROC which is 4th genaration high frequency performance Proccessor. it requaired performance FAN kit. Also Please share which RAID controller you are using. as per earlier conversation  I can see you might be upgrade RAID controller firmware. So request you to please upgrade HDD firmware also for better performance.

If you requaired more assitance, you can ask again. we will be happy to help you. 

[JensA]

Hi and thanks for the reply.
I had the wholesaler check the configuration this week, and it was more than a month ago, that the system was ordred.
They found no red flags.
The servers product ID is: P53921-B21
And the controller is a HPE SR932i-p Gen11.

The drives is: 8 SFF 24G x4NVMe/SAS UBM6 BC BP

HPE is currenctly looking for newer firmwares for the controller, there is no online that i can download, so it has to be something still in development.
I have run the newest SPP + whatever loose drivers, and firmwares i could find. I know a server has like a LOT of firmware and drivers, so finding all is not easy.

[rabindra11sharm]

Hi JensA

I am able to find performance FAN Kit part no P58465-B21. Please check once.

Also I have got a document that for useing NVME drives it requaired performance fan for dl 3X5 gen 11 systems.

Please check  maintanance guide

[ngnear]

Meanwhile, can you also share the HPE support case so we may check internally as well?

[JensA]

5381876832

[ngnear]

Thank you for sharing the case #.

I notice that the case is currently being worked upon by our engineering team.

They have currently requested some HW configuration details. Once available, they will share the further POA.

Please let me know if there is any other query for this case.

[JensA · Accepted solution]

I think i found the solution.

I moved the controller from PCI-E port 3, to 2, and i did 20 reboot tests, All normal.
Then i moved it back to PCI-E 3, and then  did 17 reboot tests, all normal.
So. The card might have needed a reseat.

[ngnear]

Thank you for the response.

Since the issue has been resolved, we request you to please update the Support case as well. 

Please feel free to reach us if any further assistance is required.

[JensA]

I did tuesday, but they still havent responded.

The support quality is verry verry bad.

[jan10]

I have the exact same issue on my servers. 68 to 72 percent. HPE support is completely useless.
I tried to solve this with them for about 6 months and completely gave up.
They cancelled my tickets 4 times without appeal, then cloning the tickets the whole cycle of level 1 support, update firmwares and logs began anew. 

It's about the 1gbps network card by Intel and any other "3rd party" device but OFFICIALLY SOLD AND SUPPORTED BY HPE!!! 
Try taking those cards out. It will instantly be -40% lower. I guarantee it. 

For you, at least you have an option to swap the intel card to the Broadcom one, which works. I dont have that luxury, as on Gen10 this **bleep** card is the only one supported. 

They wont fix this, they dont care. 

[rabindra11sharm]

Dear  jan10 

This conversation is already resolved. If you wants any help and support from us, Please open a new discussion, Please elaborate the issue what actually you are facing. share your device information and IML log snaps. to check the issue. We will deffinately address you, and help to recover your server issue.

Hope,  I could provide you with clear and helpful instructions. If you have any more questions or need further assistance, don't hesitate to ask. I'm here to help!

Have a great day!

[JensA]

Hi.

We had the same, and we removed the NIC's and changed them to other HPE NIC's.
The issue is the communication bus, that can't delever temperature information, and therefore the system will just go bananas to make sure everything is OK.

/Jens.