ML350 Gen10 Dropping Packets

[Jases]

I have a ML350 Gen10 with a HPE Ethernet 1Gb 4-port 369i Adapter.  Driver version is Intel 1.9.230.0. (Latest)  Operating system is Server 2019.  

Here's the issue:

This server runs perfectly unless I try to connect a second NIC port.  As soon as we do, the server starts dropping packets on the OSs primary port.  Even if we disable the second port, the server still drops packets.  

We have tried multiple different switches.  HP has replaced the motherboard (the NIC is integrated) and the problem persists.  HP is at the point that they want to take the server and completely replace it but this is a production server.  

Here is everything we have done to this point:

1. Ensured the latest driver was installed.
2. Moved the network connections into another switch.
3. Ran latest SPP on the server.
4. Disabled all Hyper Vs on the server.
5. Disabled the Hyper V virtual switch.
6. Disabled the 3 unused ports on the server.
7. Contacted HP and had the motherboard replaced.
8. Ran current SPP again.
9. Downloaded and installed all Windows updates.
10. Connected a USB network adapter. (This adapter worked flawlessly.)
11. Found that the NIC port that was originally configured for Hyper V was still physically plugged into the switch.  Unplugged it from the switch and the active NIC port immediately started performing properly. 

[Kamalji]

You may check this customer to set the port speed and check .

https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00038759en_us

[tomschwenz]

Hello

Did you ever get a resolution to this? I now have 3 clients that I just setup ML350 Gen10's for. All are as Hyper-V host servers. The Virtual machines do not drop a single packet and absolutly no delay connecting to shares or applications on these virtual machines but the Host OS drops packets left and right and keeps kicking me out of my remote programs. The only common thing I saw was that these servers worked perfect when we were setting up, we were only using 1 Nic in our lab and all of the issues started when we plugged in at customer location and setup all 4 NICs. I have tried upgrading\downgrading NIC drivers. Normally this wouldnt bother me since there is nothing on the host except hyperv but one of the 3 I have backups on Host server to a cloud replica server and its throwing many disconnection notices. 

[Jases]

No, as a matter of fact, I reloaded the server today and it is still doing the same thing. So I started messing around with NIC combinations. I noticed #1 would work unless #2 was plugged in. however, if 3 or 4 were plugged in #1 would work fine. That said if both 3 and 4 were plugged in #3 would start dropping. The other thing I did was plug in #1 to one switch and then #2 into another switch that wasn’t connected to anything. #1 kept up fine until I connected the second switch to the first one. At that point, #1 started dropping packets again. HP has told me they are ready to replace the server. They have already replaced the motherboard.

[Jases]

See my response below.  I'm not sure if it will notifiy you.  Let's try to put our heads together and figure this out.

[tomschwenz]

Unfortunetly I cant test anything really. I have the server setup at clients location and all is working OK as far as they can tell as the virtual Nics dont drop a single packet. Tommorow i will see if the the client can unplug 2,3,4 and ill bind my virtual Nics to 1. This is a small network, less than 15 users so i dont see any issue with running on 1 NIC for 3 virtual machines. I have also tried 2 brands of switches, Right now its on a brand new 24 port HPE switch. Everything worked fine in our lab with the virtuals and this same switch. In our lab we only had the 1 nic plugged in, i didnt think to test all of them. Started dropping the second we plugged all nics in at client site. I have triend new Intel drives and also downgraded to 3 different versions. 

Do you happen to have hyperv enabled on your server? 

I have setup 30+ setups in the past year. All ML350 Gen 10, servers 2016 or 2019, all hyperv and all Nics plugged in. Silver 4110's. Only an issue with 3 sites. 1 is server 2019 and 2 are server 2016.

[Jases]

HP contaced me and gave me this link - https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_3afcb5e2c52c4423bfe48ce5a4#tab1 

I downloaded and installed this firmware for the NIC and so far it seems to have taken care of my issue.  

[NahuTKN]

People, regarding this issu, dont forget to check the power profile in the RBSU
If using Esxi YOU should choose virtualization Power efficient.
That should correct the problem in normal conditions. Trust me

[andwork]

dec 2023 and still have that issue.

vmware esxi 8.02 installed with HPE DVD.
Power profile is: virtualization - power efficent

all firmware updated with support pack 2023.09, plus manually installed bios, controller, IE, SPS firmware from HPE website updated to TODAY: bios 3.00 - updated network card firmware to version 3.22 - smart array 6.22 rev B

https://support.hpe.com/connect/s/softwaredetails?softwareId=MTX_642243c8c63f401480425fe450&tab=Installation+Instructions

still have same issue with packet loss is using more than 1 network card. Installing an additional PCIe network card will solve the problem, so the issue is with the intel i369 adapter.

please HPE SOLVE THAT PROBLEM ONCE FOR ALL.

[Sunitha_Mod]

Hello @andwork,

Thank you for writing to us! 

We would suggest you use the below links and contact the support team so that they will check and assist you further: 

https://www.hpe.com/psnow/doc/A00039121ENW

https://support.hpe.com/hpesc/public/docDisplay?docId=cep-help_en_us&page=index.html