topic Re: Blade System Infiniband Switch Problem in BladeSystem - General

Blade System Infiniband Switch Problem

petetx01 — Mon, 23 Aug 2010 13:21:07 GMT

We have installed an HP blade system using 16 BL460c G6 blades, each with a Mellanox mezzanine card. Also in the c-7000 rack 2 cisco switch modules and 2 Qlogic Infiniband switch modules.

We are running Red Hat 5.4 with drivers downloaded from Mellanox website.

Problem is that the blades do not appear to connect with the Qlogic switch.

We have a connection between an external workstation and the c-7000 Infiniband switch which appears to work.

It appears the the Mellonax mezzanine cards are not compatible with the Qlogic switch.

Does anyone have any ideas on this?

Re: Blade System Infiniband Switch Problem

David Claypool — Mon, 23 Aug 2010 14:14:37 GMT

You need a subnet manager, either supplied from an off-board IB switch or a management module for the Qlogic switch, p/n 505959-B21

http://h18004.www1.hp.com/products/blades/components/infiniband/questionsanswers.html#b3

http://h18004.www1.hp.com/products/quickspecs/12586_div/12586_div.html

Re: Blade System Infiniband Switch Problem

petetx01 — Mon, 23 Aug 2010 16:14:10 GMT

The SM was part of the Mellanox driver pkg. We have opensm-3.3.5_mlx-0.1 installed. also installed the OFED drivers.

The question I have is the Mellanox mezzanine card compatible with the c-7000 Qlogic switch? In theory yes, but ....

Re: Blade System Infiniband Switch Problem

Heiner Billich — Tue, 24 Aug 2010 05:39:06 GMT

Hello,

just to make sure: Did you check that the IB mezzanines are in the right port on the blades, i.e. that the physical placement is right? We once did it wrong and obviously had no connection at all.

Regards,
Heiner

Re: Blade System Infiniband Switch Problem

petetx01 — Tue, 24 Aug 2010 12:14:09 GMT

Yes, we installed according to HP docs to connect to the IB switch in bay 7-8. The Onboard administrator shows connections to the switch in bay 7-8. We had Qlogic boards working but when we changed to Mellanox boards there is no communication to the Raid controller, which plugs into the switch externally, even though the OA shows connectivity to the switch.