One of our resident deal spotters, dba on the forums, spotted an amazing deal today. He found dual port Mellanox ConnectX-3 VPI cards on ebay for $299.99 each. For those who are wondering, they are the Mellanox MCX354A-FCBT models that normally retail for around $1,000 each making these cards about 1/3 the cost of purchasing new ones.
The Mellanox MCX354A-FCBT cards utilize the ConnectX-3 4099 controller and a PCIe 3.0 x8 bus to deliver a ton of bandwidth. The dual network ports are QSFP ports which is important as generally QSFP connections cannot be (easily) field terminated. That means one must judge cable lengths and purchase the correct size cables instead of just buying spools of fiber for example.
This is also one of the Mellanox 56gbps FDR Infiniband cards that does not just ultra fast Infiniband but also Ethernet. The previous generation, and very popular, QDR (40gbps) Infiniband VPI cards could handle either Infiniband or 10Gb Ethernet connections. The Mellanox MCX354A-FCBT handles not just the faster FDR spec, but also 40GbE connections. In terms of 40GbE adapters, at $300 these cards are less expensive than most 40GbE single port adapters that cannot handle Infiniband.
Of course, these cards utilize more expensive switches, but can be configured for point-to-point connections. As always do conduct your research prior to purchasing. Here is a link to the Mellanox ConnectX-3 information PDF that goes into detail about these cards. Also, feel free to use the forums to ask users questions as needed about these Mellanox 56gbps cards. We have a great community that is ever more knowledgeable about Infiniband technology.
Again here is a link to the deal (sold out) and here is a link to an ebay search for the part. Update: the original quantity was sold out but the seller listed 32 more cards here.
[tabgroup][tab title=”Mellanox MCX354A-FCBT Quick Features”]
- Virtual Protocol Interconnect
- 1μs MPI ping latency
- Up to 56Gb/s InfiniBand or 40 Gigabit Ethernet per port
- Single- and Dual-Port options available
- PCI Express 3.0 (up to 8GT/s)
- CPU offload of transport operations
- Application offload
- GPU communication acceleration
- Precision Clock Synchronization
- End-to-end QoS and congestion control
- Hardware-based I/O virtualization
- Fibre Channel encapsulation (FCoIB or FCoE)
- Ethernet encapsulation (EoIB)
[/tab][/tabgroup]
I bought some for our lab. Great find.
Edith V, could you give Hardware/Software description required which you had for 40GbE Dual Port ConnectX-3 VPI? I made some test on PC with Asus motherboards with PCIe Gen. 3 x8 on Windows 7,10, Server 2012 but the TCP throughtput between neighbour connectrors was not more than 15 GBit/s. Thank you.