Keith C. Perry on 15 Sep 2015 08:59:35 -0700

[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] Two node Infiniband connection hardware

Just some initial findings...

There are loopback QSFP heads...||106005)&sType=z

Which means there is a valid construction for a crossover cable.  You just wire "half" of the connections to another connector instead of to the same connect, similar to an ethernet or T1 signaling crossover cable.  For example, here is one of the loopback wirings for one of the molex connectors above.

This is an interesting idea so I'd like to know how you make out with it.  I initially used to build backup networks this way but even at 100Mbs the performance wasn't consistent since not all NICs are created equal and without the switch to do proper bandwidth management, packet collisions fired the back off algorithms too much.  You can play with that in software but even back then, small 100Mbs switches were a relative cheap and easy fix thus became because my protocol to dedicate a switch to the backup network (and these days that works beautifully for critical DRBD deployments).

IB is a different animal- it is built as a switching fabric so in my mind there should be mechanism present to allow without a dedicated switch.  I seem to remember that that was a primary motivation and use for it.

IB cards appear to be cheaper than 10Gbs ethernet cards too.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Keith C. Perry, MS E.E.
Owner, DAO Technologies LLC
(O) +1.215.525.4165 x2033
(M) +1.215.432.5167

From: "K.S. Bhaskar" <>
To: "Philadelphia Linux User's Group Discussion List" <>
Sent: Tuesday, September 15, 2015 10:47:19 AM
Subject: [PLUG] Two node Infiniband connection hardware

Is it possible to make a two node Infiniband connection with Infiniband cards in the two servers and a cable between them, or is a switch always required? There are hints (e.g., and suggesting it is possible), but I can't find anything definitive. Vendors of course want one to purchase a switch.

Thank you very much.

-- Bhaskar

Philadelphia Linux Users Group         --
Announcements -
General Discussion  --
Philadelphia Linux Users Group         --
Announcements -
General Discussion  --