Infiniband Mellanox IB EDR v2 36p Switch configuration for InfiniBand HDR100/Eth Adapter

[AWR]

Hi,

I have servers with P23665-H21 HPE InfiniBand HDR100/Ethernet 100Gb 1-port QSFP56 PCIe4 x16 MCX653105A-ECAT Adapters that I am trying to connect to a 834978-B22 Mellanox IB EDR v2 36p Managed Switch with P06149-B24 HPE InfiniBand HDR 200Gb QSFP56 to QSFP56 2m Direct Attach Copper Cables. The operating system of the servers is uBuntu 20.04.5. The links stay in the initialiasing state.

Any suggestions on how to resolve this?

Kind regards

Andrew Rycroft 

[Suman_1978]

Hi,

Please make sure all these components are compatible with each other.  You are combining HDR and EDR togeather.

https://www.hpe.com/psnow/doc/a00062185enw.pdf
P23665-H21, HPE InfiniBand HDR100/Ethernet 100Gb 1-port QSFP56 PCIe4 x16 MCX653105A-ECAT Adapter
Notes: These adapters are only supported on HPE ProLiant Gen10 Plus servers.
P06149-B24, HPE InfiniBand HDR 200Gb QSFP56 to QSFP56 2m Direct Attach Copper Cable

https://www.hpe.com/psnow/doc/a00016643enw.pdf
834978-B22, Mellanox InfiniBand EDR 100 Gb/sec v2 36-port Power-side-inlet Airflow Managed Switch

Switch Specifications
• Uplink ports support QSFP28 passive copper and active optical cables

Connectors and cabling
• QSFP28 uplinks ports support

Also, please check this document, HPE Compute transceiver and cable hardware matrix.

Can you please confirm the server models you are using with this setup?

Thank You!
I work with HPE but opinions expressed here are mine.
Recent Support Video Releases

[AWR]

Thank you for your response,

Yes, we do have Gen10+ servers:-

We need to connect them to the Infiniband Mellanox IB EDR v2 36p Switch configuration that was supplied with Gen 10 servers. The Gen 10 servers support the HPE IB EDR/EN 100Gb 1P 840QSFP28 Adptr, but the Gen10+ servers do not.

We need advice on how to configure the HDR100 adapters with the EDR switch. It is a bit of a puzzle.

Any input appreciated.

[Suman_1978]

Hi,

As per the product QuickSpecs, the HDR and EDR togeather may not support.

If still help is needed, please contact a HPE Reseller or HPE Support.

Thank You!
I work with HPE but opinions expressed here are mine.
Recent Support Video Releases

[AWR]

Thank you for your response, the Mellanox brochure does say:-

ConnectX-6 200Gb/s InfiniBand and Ethernet (VPI) Network Adapter Offering world-class performance for both InfiniBand and Ethernet, ConnectX-6 delivers 200Gb/s throughput with 200 million messages per second at under 600 nanoseconds of latency end-to-end. Plus, as with all Mellanox standard-based products, ConnectX-6 is backward compatible,
supporting HDR, HDR100, EDR, FDR, QDR, DDR and SDR InfiniBand and 200, 100, 50, 40, 25, and 10 GbE.

But the information on how to verify the operation is a bit thin.

Kind regards

Andrew Rycroft