BladeSystem Virtual Connect
cancel
Showing results for 
Search instead for 
Did you mean: 

NC543i (in 2x220c G7) & SR-IOV?

 
HaroldS
Visitor

NC543i (in 2x220c G7) & SR-IOV?

3 REPLIES
Hongjun Ma
Trusted Contributor

Re: NC543i (in 2x220c G7) & SR-IOV?

I did some quick check and it looks like neither SL390 nor BL220 server BIOS plan to support SR-IOV.

 

You may want to reach out to your HP account team or partner to confirm this roadmap info.

My VC blog: http://hongjunma.wordpress.com



HaroldS
Visitor

Re: NC543i (in 2x220c G7) & SR-IOV?

Thanks for answering!

Actually, I did some quick check and have succesfully used SR-IOV in my 220s' ethernet NICs (NC362i).

# lspci -v
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
    Subsystem: Hewlett-Packard Company NC362i Integrated Dual Port BL-c Gigabit Server Adapter
    Flags: bus master, fast devsel, latency 0, IRQ 24
    Memory at fbfe0000 (32-bit, non-prefetchable) [size=128K]
    Memory at fbfc0000 (32-bit, non-prefetchable) [size=128K]
    I/O ports at 3000 [size=32]
    Memory at fbfb0000 (32-bit, non-prefetchable) [size=16K]
    [virtual] Expansion ROM at f4000000 [disabled] [size=128K]
    Capabilities: [40] Power Management version 3
    Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
    Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
    Capabilities: [a0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [140] Device Serial Number 68-b5-99-ff-ff-b0-73-f0
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
    Kernel driver in use: igb
    Kernel modules: igb
(...)
03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    Subsystem: Hewlett-Packard Company Device 31ff
    Flags: bus master, fast devsel, latency 0
    [virtual] Memory at fbf00000 (64-bit, non-prefetchable) [size=16K]
    [virtual] Memory at fbf20000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
    Capabilities: [a0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Kernel driver in use: igbvf
    Kernel modules: igbvf
(...)

 

VFs passed to KVM VMs seemed to work fine ;-)

We've got some apps that use RDMA and I would like to put them in VMs. AFAIK Mellanox chips used in those cards are capable of SR-IOV, but it's not exposed (I believe because of firmware):

02:00.0 Network controller: Mellanox Technologies MT26438 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE Virtualization+] (rev b0)
    Subsystem: Hewlett-Packard Company NC543i 1-port 4x QDR IB/Flex-10 10Gb Adapter
    Flags: bus master, fast devsel, latency 0, IRQ 30
    Memory at fbe00000 (64-bit, non-prefetchable) [size=1M]
    Memory at fa000000 (64-bit, prefetchable) [size=8M]
    [virtual] Expansion ROM at f8800000 [disabled] [size=1M]
    Capabilities: [40] Power Management version 3
    Capabilities: [9c] MSI-X: Enable+ Count=256 Masked-
    Capabilities: [60] Express Endpoint, MSI 00
    Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [148] Device Serial Number 78-e7-d1-03-00-03-9b-c0
    Kernel driver in use: mlx4_core
    Kernel modules: mlx4_core, mlx4_en

so enabling SR-IOV in driver just cause an error:

mlx4_core 0000:02:00.0: Enabling sriov with:1 vfs
mlx4_core 0000:02:00.0: Failed to enable sriov, aborting.

 

 

HaroldS
Visitor

Re: NC543i (in 2x220c G7) & SR-IOV?

OK, I have borrowed [;-)] one SL390s G7 and burned latest firmware on its IB adapter. Does anyone know how to enable SR-IOV?

[root@cuda ~]# mstflint -d 05:00.0 q
Image type:      ConnectX
FW Version:      2.9.1000
Rom Info:        type=PXE  version=3.3.500 devid=26438 proto=VPI
Device ID:       26438
Description:     Node             Port1            Port2            Sys image
GUIDs:           78e7d1030024d764 78e7d1030024d765 78e7d1030024d766 78e7d1030024d767
MACs:                                 78e7d124d764     78e7d124d765
Board ID:         (HP_0200000003)
VSD:             
PSID:            HP_0200000003
[root@cuda ~]# mstflint -d 05:00.0 dc
;; Generated automatically by iniprep tool on Tue Jul 19 18:13:04 IDT 2011 from ./b0_hp_ariston.prs
;; Generated automatically by iniprep tool on Sun Aug 16 17:58:41 IDT 2009 from ./b0_raptor.prs
;;
;; PRS  FILE FOR Raptor
;; $Id: b0_hp_ariston.prs,v 1.23.2.3 2011-07-18 07:17:07 danb Exp $




[PS_INFO]
Name = CX2_Ariston1
Description = ConnectX-2 VPI PCIe 2.0 5GT/s - IB QDR / 10GigE Virtualization+

[ADAPTER]
PSID = HP_0200000003
pcie_gen2_speed_supported = true
adapter_dev_id = 0x6746
silicon_rev = 0xb0

gpio_mode1 = 0x80000
(...)
[root@cuda ~]# mstflint -d 05:00.0 dc | grep -i iov
[root@cuda ~]# lspci -s 05:00.0 -v
05:00.0 Network controller: Mellanox Technologies MT26438 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE Virtualization+] (rev b0)
        Subsystem: Hewlett-Packard Company NC543i 2-port 4xQDR IB/10Gb Adapter
        Flags: bus master, fast devsel, latency 0, IRQ 26
        Memory at ebe00000 (64-bit, non-prefetchable) [size=1M]
        Memory at dd800000 (64-bit, prefetchable) [size=8M]
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Vital Product Data
        Capabilities: [9c] MSI-X: Enable+ Count=128 Masked-
        Capabilities: [60] Express Endpoint, MSI 00
        Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [148] Device Serial Number 78-e7-d1-03-00-24-d7-64
        Kernel driver in use: mlx4_core
        Kernel modules: mlx4_en, mlx4_core