Storage Spaces Direct with

[richsmif76]

I have a HP FlexFabric 2 port 10GB 556FLR-SFP+ and FlexFabric 557SFP+ 10Gb in a DL380 G9.

4 of these servers are configured with 2016 DC and trying to get decent throughput on S2D.  Write speed from 1 volume to another is 150Mb/s.  I have 52x 1.2TB SAS drives and 12x 800GB SSD.

I have rebuilt the servers from scratch, and trying to configure QoS, when i configure Get-NetAdapterQos, nothing is returned.  I have run the latest SUM (Aug 17) so technically should have the latest drivers.

Originally i configured with VMM, I've lost complete faint in this setup as the networking performance is dreadful.

Using HP 5700 switches, 2x of them.

Please help.

[richsmif76]

The other problem i have is Enable-NetAdapterQos does not work.  Says it cant find it?????????

PS C:\Windows\system32> Get-NetAdapter -InterfaceAlias "embedded flex*" | ft name

name
----
Embedded FlexibleLOM 1 Port 2
Embedded FlexibleLOM 1 Port 1

PS C:\Windows\system32> Enable-NetAdapterQos -InterfaceAlias "Embedded FlexibleLOM 1 Port 2"
Enable-NetAdapterQos : No MSFT_NetAdapterQosSettingData objects found with property 'Name' equal to 'Embedded FlexibleLOM 1 Port 2'.  Verify the value of the property and retry.
At line:1 char:1
+ Enable-NetAdapterQos -InterfaceAlias "Embedded FlexibleLOM 1 Port 2"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Embedded FlexibleLOM 1 Port 2:String) [Enable-NetAdapterQos], CimJobException
    + FullyQualifiedErrorId : CmdletizationQuery_NotFound_Name,Enable-NetAdapterQos

PS C:\Windows\system32>

[richsmif76]

Having raised the issue with Microsoft directly, it seems our network cards are not on the supported list even though they support SMB Direct.

Can anyone confirm if they are tried S2D on the following cards, or what settings they used?

FlexFabric 556FLR-SFP+

FlexFabric 557-SFP+

[skillful]

Same message error when I try Enable-NetAdapterQos 

I'm using HPE FlexFabric 10Gb 2-port 556FLR-SFP+ Adapter which is RDMA(RoCE) compatible.

Did you get any feedback from Microsoft ?

[Aurock]

I'm also running into the same issue with the get-netadapterqos and enable-netadapterqos commands, using the same 556FLR-SFP+ adapters. 

Did either of you find a solution for this, or a workaround?

I had previously configured my adapters for storage spaces using the Emulex OneCommand Manager utility, and between that and the switch had configured much of the QOS stuff before.  I wonder if the issue is just that the driver for this adapter doesn't support configuration with these powershell commands, but can be configured properly using other tools.  

[richsmif76]

I never found a solution to this and thus the S2D cluster was never stable enough to go into production.  I had multipul tickets raised with Microsoft to find a solution.  Nodes BSOD randomly, applied patches (even undistributed patches) - November 2017.  There are saying its a driver related issue and are ploughing through blue screen memory dumps.

Combine that with S2D been pulled from 2016 v1709, erm no thanks.  Plan is to reconverge the hardware in 3 years when the current cluster becomes a DR, and S2D would of been out for few years by then.

We have since lost all confidence in S2D as a solution and presently purchasing a 2x node G10 360 SOFS cluster with 3x D3710 chassis directly attached.  Ripping all the storage out of the 4x nodes and putting it into D3710's.  Ditching the 556FLR's and replacing with 640FLR Mellanox through out.  The 556FLR will be repurposed elsewhere in less intensive workloads and go towards switch decommissioning.

RIP S2D (for now).

[HP-Schrauber]

Hello,

can you try this Command in Powershell (Administrator Mode) : Get-NetAdapter | fl and poste the Output here if any.

Regards

M.Fuhr

[HP-Schrauber]

Hello,

i have a little Script for the Storage Space Direct konfiguration on Server 2016 Datacenter Nodes.
In my Case the Adapter Name are : 10GBPort1 and 10GBPort2 my Hardware is one HP 546FLR-SFP Card.

Script for Konfigure the QOS etc.

# Clear previous configurations
Remove-NetQosTrafficClass
Remove-NetQosPolicy -Confirm:$False

# Enable DCB, if it’s not already done through the wizard.
Install-WindowsFeature Data-Center-Bridging

# Disable the DCBx setting:
Set-NetQosDcbxSetting -Willing 0

# Create QoS policies and tag each type of traffic with the relevant priority
New-NetQosPolicy "SMB" -NetDirectPortMatchCondition 445 -PriorityValue8021Action 3
New-NetQosPolicy "Cluster" -Cluster -PriorityValue8021Action 5
New-NetQosPolicy "DEFAULT" -Default -PriorityValue8021Action 3
New-NetQosPolicy "TCP" -IPProtocolMatchCondition TCP -PriorityValue8021Action 1
New-NetQosPolicy "UDP" -IPProtocolMatchCondition UDP -PriorityValue8021Action 1
New-NetQosTrafficClass "SMB"  -Priority 3  -BandwidthPercentage 60  -Algorithm ETS
New-NetQosTrafficClass "Cluster"  -Priority 1 -BandwidthPercentage 5 -Algorithm ETS


# If VLANs are used, mark the egress traffic with the relevant VlanID:
Set-NetAdapterAdvancedProperty -Name <Network Adapter Name> -RegistryKeyword "VlanID" -RegistryValue <ID>

# Enable Priority Flow Control (PFC) on a specific priority. Disable for others
Enable-NetQosFlowControl -Priority 5
Disable-NetQosFlowControl 0,1,2,3,4,6,7

# Enable QoS on the relevant interface (named here: RoCE_1)
Enable-NetAdapterQos -InterfaceAlias "10GBPort1"
Enable-NetAdapterQos -InterfaceAlias "10GBPort2"

# Enable Jumbo Frames auf Netwerkkarte aktivieren mit einen MTU von 9014
Set-NetAdapterAdvancedProperty -Name "10GBPort1" -RegistryKeyword '*JumboPacket' -RegistryValue '9014'
Set-NetAdapterAdvancedProperty -Name "10GBPort2" -RegistryKeyword '*JumboPacket' -RegistryValue '9014'
Ping -f -l 8000 192.168.0.2 # Jumbo Frame Test

Regards
M.Fuhr