Supermicro MegaDC ARS-211M-NR Internal Hardware Overview
The system follows a simple front-to-back airflow design. Four large fans move air through the system. It is fun that by today’s standards, a 2U system that uses well under 800W without GPUs installed is considered low density.
Something you will note here is that the tool-less risers are long, and go all the way across the CPU and memory area and to the fan partition. These have pins that also help keep the airflow guide that goes over the CPU and memory in place. No screws!
This airflow guide is actually awesome. We review a ton of servers, and this one is particularly cool. It is a stacked two-part airflow guide where the top part can be customized if the system has rear GPUs installed. This one was surprisingly easy to install, was held in place by the riser’s pins, and has that flexibility built-in. This is a big generational improvement.
Now what everyone probably wants to see, the CPU and memory area.
This is an Ampere AmpereOne Socket LGA5964 platform.
That means we have a 192 core AmpereOne A192-32X processor installed.
Some Ampere Altra/ Altra Max platforms got CPU carriers to help installation but the first few Altra platforms had some of the scariest CPU installations we have done recently, albeit ones that we completed successfully for years. The AmpereOne processor hear has a carrier that helps place and align the CPU in its socket. In some ways, this is similar to an AMD EPYC by using the socket carrier instead of the Intel Xeon thermal solution mounting, but it goes directly onto the socket, unlike AMD’s solution.
Here is the Supermicro R13SPD motherboard’s LGA5964 socket in all of its glory.
This is really a cloud-native processor, so it competes with Intel’s Sierra Forest E-core line. Here is the much larger 192-core AmpereOne to the smaller Intel Xeon 6700E.
On the AMD side, the closest competitor for the next few days is probably Bergamo, or the AMD EPYC 9754 shown here. Ampere is much closer in package size to the AMD EPYC 9000 series and the Intel 6900 series socket. Hopefully, one day we get a chip we can use like our Altra Max for photos to show a direct size comparison. Our Granite Rapids-AP system is very sensitive, so we have been a bit more cautious about opening it up and pulling CPUs for photo shoots.
Something that might be apparent is that unlike Intel and AMD CPUs, the bare compute die is exposed with a heat spreader frame around the outside for the PCIe and memory dies on Ampere one. As a result, the cooler design is cooling the CPU die in the center, and then a higher outer part for the PCIe and memory controller dies.
Here is another look at that two-stage solution.
This is the 8-channel AmpereOne platform. We covered how there will be a 12-channel platform that will scale to 256 cores coming.
This is an 8-channel DDR5 platform, so we have DDR5-5600 memory installed. We can see that this CPU supports 2DPC operation. The AmpereOne chip only supports DDR5-5200, so that is the speed this memory is running at. Using 2DPC mode, we can get up to 4TB of capacity, but the speed decreases to DDR5-4400 speeds.
Next to the memory, we get an M.2 slot.
Moving to the back of the R13SPD motherboard we have a PCIe riser slot.
The heatsink in the middle is a Broadcom BCM57414 dual port 25GbE network controller, not a PCH as we would have seen on older Intel Xeon systems.
In the center, we have another PCIe riser slot.
That riser we showed earlier allows a dual-slot GPU to be installed over a low-profile PCIe Gen5 x16 slot and the AIOM/ OCP NIC 3.0 slot.
One little quirk of this system is that due to the placement of the front GPU slots, the fan on the side of the system with the two front GPU slots and power supplies is recessed to make room.
Next, let us get to the system topology and block diagram.
That’s great performance per dollar at least
I don’t get why you’d want a 1S not a 2S for these. If you’re trying to save money, then 2S 1 NIC shares common components except CPUs and mem so that’s much cheaper.
At idling Ampere’s power consumption seems/feels “broken” – Phoronix in its latest benchmarks with EPYC 5c does too, make this same observation.