topic Re: DL380G9 - Network Troubles in ProLiant Servers (ML,DL,SL)

DL380G9 - Network Troubles

Patrick Neuner — Mon, 14 Sep 2015 09:00:38 GMT

Hello,

we got a shiny new DL380 Gen 9 Server - Windows 2012R2 (Hyper-V Role) installed on it.

Networking through HP Ethernet 1Gb 4-port 331i Adapter.

Server is connected to ProCurve Switch 3400cl-48G through 4 Cables (NIC-Teaming) - had it conected with only 1 Cable before - got same Problem. (Server/Switch - all latest Firmware/Drivers)

Once in a while the Server loses Network Connectivity - physical Link stays up but the connection drops the Gateway ...

When querying the ARP-Cache on the Server it shows me:

Gateway-IP-Address 00-00-00-00-00-00 invalid

So it lost the MAC of the Gateway ... ?!?

When I plug out/in the Cable - all work's without problems again. (Can also just disable Switch-Port and re-enable it)

Already searched hours and hours for a solution - but didn't find anything ... Hope someone here can help me with this odd problem?!

Thank you

Andreas Schnederle-Wagner

Re: DL380G9 - Network Troubles

Patrick Neuner — Mon, 14 Sep 2015 09:24:25 GMT

some more Information:

cmd - NETSH INTERFACE IPv4 SHOW NEIGHBORS LEVEL=VERBOSE:

     Nachbar 10.2.0.254
     ----------------------------------------------
     Datenverbindungsadresse    : 08-5b-0e-97-3b-0e
     Status               : Abgelaufen
     Letzte Aktualisierung         : 7

cmd - arp -a:

10.2.0.254 08-5b-0e-97-3b-0e dynamisch

Network still working - even it shows expired? Will output again when Network won't work anymore ...

Re: DL380G9 - Network Troubles

Patrick Neuner — Mon, 14 Sep 2015 15:41:51 GMT

alright - Network isn't working again ...

- 10.2.0.254 = Gateway IP

- 10.2.0.27 = Hyper-V Host

- 10.2.0.115 = Virtual Machine

- 10.2.1.14 = my Workstation

cmd - NETSH INTERFACE IPv4 SHOW NEIGHBORS

10.2.0.254 Nicht erreichbar Unvollständig

10.2.0.14 Nicht erreichbar Nicht erreichbar

10.2.0.115 00-15-d5-00-1b-00 Erreichbar

arp -av

10.2.0.254 08-5b-0e-97-3b-0e ungültig

10.2.0.14 2c-41-38-ba-de-c4 ungültig

10.2.0.115 00-15-d5-00-1b-00 dynamisch

all 4 physical Links are UP, NIC-Teaming also shows all 4 LOMs online

From my Workstation I can ping a Virtual Machine running on this Hyper-V Host but not the Hyper-V Host itself.

From within the Hyper-V Host I can also ping the Virtual Machine - but I can't ping my Workstation.

From within the virtual Machine I can ping everything - the Hyper-V Root Server, my Workstation and even the Internet ...

So only Network connectivity from Hyper-V Host is impacted.

Not really sure where to look for this Problem?!?!

Re: DL380G9 - Network Troubles

Patrick Neuner — Mon, 14 Sep 2015 16:05:18 GMT

some more information:

From switch:

ProCurve Switch 3400cl-48G# show arp

IP ARP table

IP Address      MAC Address       Type    Port
--------------- ----------------- ------- ----
10.2.0.25       001e67-7ae2d4     dynamic Trk1
10.2.0.27       3ca82a-227980     dynamic 13
10.2.0.115      00155d-001b00     dynamic 14
10.2.0.254      085b0e-973b0e     dynamic 1
10.2.1.11       b4b52f-cca2b3     dynamic 31
10.2.1.14       2c4138-badec4     dynamic 30

10.2.0.115 - the virtual machine which has full network connectivity is on Switch Port 14

10.2.0.27 - Hyper-V Root on Port 13

As soon as I disable Port 13 the NIC Teaming switches to another Switch Port and everything is working again.

But after a random Time the same behaviour also happens on another Switch-Port - so I guess it's not a faulty Switch-Port ...

Re: DL380G9 - Network Troubles

Patrick Neuner — Wed, 16 Sep 2015 10:21:12 GMT

Update: Since I disabled VMQ/Large Send Offload V2 on all LOMs yesterday -no more Network Troubles ...

Executed those Commands for every LOM:

Set-NetAdapterAdvancedProperty "Embedded LOM 1" -DisplayName "Large Send Offload V2 (IPv4)" –DisplayValue Disabled

Set-NetAdapterAdvancedProperty "Embedded LOM 1" -DisplayName "vmq vlan filtering" –DisplayValue Disabled

Set-NetAdapterAdvancedProperty "Embedded LOM 1" -DisplayName "Virtual Machine Queues" –DisplayValue Disabled

What puzzles me a bit is that this known Broadcom+VMQ Problem should be solved with Broadcom Driver b57nd60a.sys Version 16,8 or higher as far as I know? (HP Changelog / Broadcom Driver Changelog)

Any ideas on this?

Andreas

Re: DL380G9 - Network Troubles

Patrick Neuner — Thu, 17 Sep 2015 07:56:30 GMT

Still no problems ... seems like Broadcom did not fix all Problems with VMQ in their Drivers ...

Will raise a Case with Broadcom.

Andreas