ASRock Rack TURIN2D16-2T AMD EPYC 9005 CPU Support List
When this motherboard first arrived, our first thought was AMD EPYC 9755. These are high-end 128 core parts that use 500W each.
It turns out that the AMD EPYC SKUs supported are all 400W and lower, which is clearly in the specs and the CPU support list that we failed to read before trying to use higher-power CPUs.
The jump to 500W required additional power components, which again, is a challenge on a small motherboard like this.
Key Lessons Learned
At this point, saying a “small motherboard” with one that is a 12″ x 13.2″ EEB size feels strange, but that is what it is. When we got to the block diagram, and then dug into the system, it became very apparent how much work the ASRock Rack engineers had to do just to make this work. When we talk about radical motherboard designs, this feels like the ASRock Rack ROMED4ID-2T mITX-ish AMD EPYC platform in terms of the hoops that the designers had to go through to make it work.
This platform is also a great one to illustrate many of the industry’s design challenges that we have discussed on STH with newer PCIe Gen5 designs and large CPUs.
Final Words
To many browsing the Internet, this looks like a fairly standard dual socket motherboard. What we know is that this almost should not exist. But for the engineering prowess and determination of some folks at ASRock Rack this would not exist. It is pushing the bounds of what can be done in a more traditional motherboard form factor.
All of that work means that the ASRock Rack TURIN2D16-2T might just be the most feature rich dual AMD EPYC 9004/ 9005 EEB motherboard out there.
PCB tracing also responsible for MCIO and triple power connectors effectively swapping places?
Similar looking to the Gigabyte MZ73-LM2, except that’s 500W CPUs, 4 PCIe 5, 2 x 10Gb/s LAN, and 24 DIMMs, etc.
I have a Asrock GENOAD8X-2TBCM with an 84c Epyc, which recently died withing warrant time (doesn’t post anymore), but I had a supplier which tries to screw me over, and at the moment I have contacted my lawyer to get my money back. I have the feeling there was always something wrong with the GENOAD8X-2TBCM, I never could enter anything via remote management when the OS was booted, also I tried to install XCP-NG but not one of the NVMe were detected during install, so I installed proxmox on it which ran, but there were always random crashes.
This board looks like a great update, unfortunately it is still nowhere to be found in Europe, disadvantage would be that I need a second Epyc 84c, does anyone know if running an assymetric configuration is possible 32cEpyc and 84c Epyc, probably not right ? I’m a bit afraid if I buy this board that I also can not install XCP-NG, which is my favorite Hypervisor, since I have it running on a Ryzen 5950x and it runs for months without problems, if something crashed it is a VM but the hypervisor is running without hickups.
On the other hand I”m contenmplating a Supermicro H13SSL-NT, more memoryslots but less pcie 5.0 slots, don’t like the smaller ATX formfactor and with supermicro I had a bad experience of networkcards getting too hot and dying because of that, I’m running these systems in a FractalXL case with a lot of fans, but its not servergrade chassis with the accompanying air jetstream which Supermicro might be expecting for their boards.
@Rupi
Both CPUs need to be identical.
XCP-ng is running a relatively old kernel (4.19) so it might have problems with newer hardware. For NVMe there’s an option in BIOS to override the device firmware with BIOS firmware. It’s in Advanced -> NVMe Configuration. It might make the drives visible in older kernels.
@Kyle, ok, thanks for the info, I could run in with 1 CPU for a while although I can’t use my 100GB optical card until I buy another CPU, I expect them to drop with the release of the Turin CPU’s.
I even had that same problem installing XCP-ng 8.3, weird they would use an older kernel with their latest release. Regarding the NVME firmware, never overrode the default setting hopefully that was the problem. Of course I can’t try it on my broken board, but definitely will try it on this board if I decide to buy next week, will have to order it from the US, never did that probably some customs tariffs will be imposed, but here in Europe mainboard prices are rediculously high, so I guess that evens it out.
Hi
I’m not sure if folks noticed,
but it only has 3 QPI lanes between processors.
It does not support the higher Turins, 450w
it may have 5 PCIE slots but only 3 are at PCI Gen 5
the on board m.2 is PCI gen 4
and all in all with few exceptions on an extra port for 2 more 4x cxl or nvme express;
it seems a downgrade to the mZ73-LM0 Rev 2 or Rev 3 from gigabyte
Did I miss something?
Jay
Jay they’ve got an entire section devoted to that. That GB board’s got more DDR5 but 24 fewer PCIe lanes. Gen 4 or gen 5, you often need slots and lanes. If you’re buying a 32 core epyc then you might be connecting network and ssds but your not gonna fill 12ch 9 of 10 times.
Now THIS is what I referred to in my comments on the 48-DIMM motherboards STH covered a while back! THIS is what I talking about! Put one CPU and memory stack IN FRONT the other and then 48 DIMMs (24 per processor) won’t be such a problem to physically place and route. In fact, adapt THIS board a little by eliminating the PCIe 4.0 slots, re-arrange the other components (BMC, glue logic, I/O chips, VRMs, MCIO connections etc.) a bit and this board could even take 24 DIMMs at 12 per CPU! Re-interpret this board into a more typical board for a rackmount and the footprint will allow a full 64 DDR4 or DDR5 DIMMs, WITHOUT increasing the width OR needing to create a special proprietary board just for that purpose! You might even could get away with this even on dual LGA-7529 CPUs (which I think are a good bit larger than AMD’s socket SP5/LGA-6096)!! Not sure about the upcoming LGA-9324 for Diamond Rapids, though, as THAT CPU might be a bit too big for even this to work.
EDIT: Insert the word “was” between the “I” and “talking” into the line “THIS is what I talking about!”. I can’t proofread apparently.
@Harold, if you look at the Phoronix articles “DDR5 Memory Channel Scaling Performance With AMD EPYC 9004 Series” and “8 vs. 12 Channel DDR5-6000 Memory Performance With AMD 5th Gen EPYC” you’ll have your answer, filling the slots (and more of them) is like free cores; it can add significant speed ups, and allow you to use smaller DIMMs at a lower cost.
The 2x MCIO 8i is what’s replacing PCIe slots that are too far away (without retimers).
With that MB you still get 2 slots with one CPU, unlike the ASRock which requires 2 CPUs to be usable. I think we need to hear “why” the ASRock is “better”, not simply a declaration that it is; when available information suggests otherwise.