FS N9510-64D 64-port 400GbE Switch Internal Overview
Here is an overview of the switch internals. We are going to do something a bit different this time and work from the rear (left below) to the front (right below).
Here is a quick look at one of the hot-swap fan connectors.
In this switch, there are two levels of fan control boards each powering four fan modules.
Inside the switch, there are two main baseboards.
The middle baseboard has the Intel Xeon D-1627 (4 core/ 8 thread) management processor. Next to it is a single M.2 SSD for OS storage but there is a second unpopulated slot.
Moving up to the main switch and QSFP-DD PCB, we can see Lattice has some placements here.
There is also a Xilinx AMD Artix-7 FPGA.
The big feature on this PCB is the massive heatsink that covers the main switch chip.
The main switch chip is the Broadcom Tomahawk 4. What is fascinating here is that this switch chip is socketed in a TE Connectivity LGA8371. We covered this solution more in our This is a Broadcom Tomahawk 4 64-port 400GbE Switch Chip piece.
As a quick one here, the QSFP-DD cages above can be seen with large cage heatsinks. That is one of the big differences between QSFP-DD and OSFP since OSFP usually has the heatsinks on the optical modules/ DACs.
Here is the 4th Gen Intel Xeon Scalable, codenamed “Sapphire Rapids” next to the Broadcom Tomahawk 4.
Just for some sense, here is the Dynatron J12 for the AMD EPYC 9004 “Genoa” next to the FS Tomahawk 4 heatsink.
Internal grounding seems significant between the PCBs.
Here is another photo we had of two connecting boards.
Next, let us get to putting together the network and getting everything working.
That heatsink is crazy. Makes me wonder what is going to happen (en mass) first: co-packaged optics, or water-cooled switches?
Question: How much does the Tomahawk 4 weigh?
Just musing..but is it odd a 64 port 400GbE switch (and other enterprise or service provider level equipment) is presented on a site called “Serve the HOME” Don’t get me wrong, I love the content here!
Yeah, that is an interesting switch. The thing is so big that it needs a standard server chassis just for that MASSIVE heatsink and 8,000-plus pin Broadcom Tomahawk 4 switch ASIC. Very cool. :-)
Page 1, paragraph 4: “The top 4U is used for cooling.” should read “The top 2U is used for cooling.”
Page 3, paragraph 3: “We will quickly note that while we were using Intel Xeon “Sapphire Rapids”” – missing end of sentence.
Sure hope this has a better warranty than “30 days then we send it back to China for a few months” I experienced with an obviously much less expensive switch.
as mentioned before, use the 2x200G cx-7 nic and a cheap ($150) QSFP-DD to 2 x QSFP56 DAC.
Don’t waste your time with the flat top OSFP weirdness!
@Kwisatz: It’s definitely a misleading site name and it causes so much confusion it’s an FAQ in the “about” section: https://www.servethehome.com/about/
“STH may say “home” in the title which confuses some. Home is actually a reference to a users [sic] /home/ directory in Linux. We scale from those looking to have in home or office servers all the way up to some of the largest hosting organizations in the world.”
I do wonder how many of us ended up here because we were after information about home server equipment. I guess judging by the number of non-enterprise mini-PC and 2.5 GbE articles, quite a few of us.
I wonder how often 400Gb ports are saturated in this context – aren’t latency and rate more interesting?
It would be useful if you would cover the management features of the switch. Sure, the heatsink is cool and all but that’s not really functional or useful to users, is it?