DL380G9 - Network Troubles

[Patrick Neuner]

Hello,

we got a shiny new DL380 Gen 9 Server - Windows 2012R2 (Hyper-V Role) installed on it.

Networking through HP Ethernet 1Gb 4-port 331i Adapter.

Server is connected to ProCurve Switch 3400cl-48G through 4 Cables (NIC-Teaming) - had it conected with only 1 Cable before - got same Problem. (Server/Switch - all latest Firmware/Drivers)

Once in a while the Server loses Network Connectivity - physical Link stays up but the connection drops the Gateway ...

When querying the ARP-Cache on the Server it shows me:

Gateway-IP-Address               00-00-00-00-00-00              invalid

So it lost the MAC of the Gateway ... ?!?

When I plug out/in the Cable - all work's without problems again. (Can also just disable Switch-Port and re-enable it)

Already searched hours and hours for a solution - but didn't find anything ... Hope someone here can help me with this odd problem?!

Thank you

Andreas Schnederle-Wagner

[Patrick Neuner]

some more Information:

cmd - NETSH INTERFACE IPv4 SHOW NEIGHBORS LEVEL=VERBOSE:

     Nachbar 10.2.0.254
     ----------------------------------------------
     Datenverbindungsadresse    : 08-5b-0e-97-3b-0e
     Status               : Abgelaufen
     Letzte Aktualisierung         : 7

cmd - arp -a:

     10.2.0.254            08-5b-0e-97-3b-0e     dynamisch

Network still working - even it shows expired? Will output again when Network won't work anymore ...

[Patrick Neuner]

alright - Network isn't working again ...

- 10.2.0.254 = Gateway IP

- 10.2.0.27 = Hyper-V Host

- 10.2.0.115 = Virtual Machine

- 10.2.1.14 = my Workstation

cmd - NETSH INTERFACE IPv4 SHOW NEIGHBORS

   10.2.0.254       Nicht erreichbar         Unvollständig

   10.2.0.14          Nicht erreichbar         Nicht erreichbar

   10.2.0.115        00-15-d5-00-1b-00   Erreichbar

arp -av

     10.2.0.254         08-5b-0e-97-3b-0e      ungültig

     10.2.0.14            2c-41-38-ba-de-c4       ungültig

     10.2.0.115         00-15-d5-00-1b-00     dynamisch

all 4 physical Links are UP, NIC-Teaming also shows all 4 LOMs online

From my Workstation I can ping a Virtual Machine running on this Hyper-V Host but not the Hyper-V Host itself.

From within the Hyper-V Host I can also ping the Virtual Machine - but I can't ping my Workstation.

From within the virtual Machine I can ping everything - the Hyper-V Root Server, my Workstation and even the Internet ...

So only Network connectivity from Hyper-V Host is impacted.

Not really sure where to look for this Problem?!?!

[Patrick Neuner]

some more information:

From switch:

ProCurve Switch 3400cl-48G# show arp

 IP ARP table

  IP Address      MAC Address       Type    Port
  --------------- ----------------- ------- ----
  10.2.0.25       001e67-7ae2d4     dynamic Trk1
  10.2.0.27       3ca82a-227980     dynamic 13  
  10.2.0.115      00155d-001b00     dynamic 14  
  10.2.0.254      085b0e-973b0e     dynamic 1   
  10.2.1.11       b4b52f-cca2b3     dynamic 31  
  10.2.1.14       2c4138-badec4     dynamic 30

10.2.0.115 - the virtual machine which has full network connectivity is on Switch Port 14

10.2.0.27 - Hyper-V Root on Port 13

As soon as I disable Port 13 the NIC Teaming switches to another Switch Port  and everything is working again.

But after a random Time the same behaviour also happens on another Switch-Port - so I guess it's not a faulty Switch-Port ...

[Patrick Neuner · Accepted solution]

Update: Since I disabled VMQ/Large Send Offload V2 on all LOMs yesterday -no more Network Troubles ...

Executed those Commands for every LOM:

  Set-NetAdapterAdvancedProperty "Embedded LOM 1" -DisplayName "Large Send Offload V2 (IPv4)" –DisplayValue Disabled

  Set-NetAdapterAdvancedProperty "Embedded LOM 1" -DisplayName "vmq vlan filtering" –DisplayValue Disabled

  Set-NetAdapterAdvancedProperty "Embedded LOM 1" -DisplayName "Virtual Machine Queues" –DisplayValue Disabled

What puzzles me a bit is that this known Broadcom+VMQ Problem should be solved with Broadcom Driver b57nd60a.sys Version 16,8 or higher as far as I know? (HP Changelog / Broadcom Driver Changelog)

Any ideas on this?

Andreas

[Patrick Neuner]

Still no problems ... seems like Broadcom did not fix all Problems with VMQ in their Drivers ... 

Will raise a Case with Broadcom.

Andreas