ASRock Rack GENOAD8X-2T/BCM Topology
Given just how much I/O there is on this platform, it is important to take a look at the block diagram.
While everything ultimately goes back to the single AMD EPYC CPU, several of the PCIe Gen5 lanes also have options to power the MCIO connectors and M.2 slots, but that depends on how they are used.
The other big point that one will want to know is that the NCSI connection is to the second LAN port on the Broadcom 10Gbase-T NIC. That is how the BMC communicates with the 10Gbase-T NICs and is important for some folks.
ASRock Rack GENOAD8X-2T/BCM Performance
One of the big open questions with this platform was the performance. Especially given the 8-channel memory configuration. We ran a 96-core AMD EPYC through a suite of benchmarks to gauge the impact.
Overall, the performance was good, but there was some loss simply due to having fewer memory channels available. At the same time, it is important to take a step back here. Having a bit less performance than a reference Genoa platform still means it can be as fast, or faster than many dual-socket servers from even a generation ago. What is more, this is one of the few options that has this much PCIe Gen5 connectivity exposed via PCIe slots.
One way to look at the above is, of course, shorter bars are shorter. The other way to look at this is it is a lot of performance in a platform that can be installed into many different chassis and locations.
Final Words
This is a motherboard that we absolutely love. While some may bemoan the loss of an ATX form factor, the fact is that newer CPUs need more physical motherboard space to provide PCIe I/O along with the massive processors and memory loadouts. Bryan actually tried putting together a workstation build with the ASRock Rack GENOAD8UD-2T/X550 but the lack of USB I/O was a challenge there. Having used this motherboard, perhaps we will resurrect that project. In either case, that brings us to the point of this, it is one of the few Genoa motherboards that is a standard non-proprietary form factor. That means it can be integrated into a variety of projects.
The AMD Socket SP5 is ultra-versatile. It can handle standard AMD EPYC Genoa CPUs, high-cache Genoa-X parts, and even the high-core count Bergamo parts.
Aside from having up to 128 cores / 256 threads in the single socket the PCIe Gen5 I/O is going to make a segment of our readers extremely excited. That is a TON of PCIe connectivity for a single-socket platform. As PCIe Gen5 devices become more commonplace, this motherboard has plenty of room to expand.
On the subject of the key lessons learned from that PCIe I/O is what is required to make that happen on a motherboard like this. There is a large array of Phison redrivers on this motherboard just to help with PCIe Gen5 signal integrity. We have been talking about the looming challenges with PCIe signal integrity as speeds increase for years, and this is a great example of its impact. One thing to be certain of is that this is one of the biggest features of this board. Having redrivers in the chain adds cost, but it boosts signal integrity. A cheaper option would be to skip the redrivers and just hope that devices connect properly. It is great to see ASRock Rack went for the higher-end solution on this motherboard.
It is hard to fathom that the ASRock Rack GENOAD8X-2T/BCM is an EEB-size motherboard that still had to reduce DDR5 DIMM slots to eight in order to fit everything into a non-proprietary form factor. Still, that is what we have here and the results are awesome. When we first saw this board in trade show booths, we knew we needed to show our readers it. Once we got to use it, it certainly met all of our lofty expectations for an awesome AMD EPYC powered PCIe Gen5 platform.
Please re-read and re-write the paragraph about memory slots/ bandwidth, then you can delete this comment.
“Putting 24 DDR5 DIMM slots next to a SP5 socket would have taken up an enormous amount of motherboard space so the trade-off was made.” LOL
Dope… SP5 is twelve channel… harhar. It’s been a long week!
It is time for the PCIe x16 slot to DIE. We don’t have room on the mobo and I challenge anyone to find 7 useful single slot cards to plug into this thing.
We should have moved to cabled PCIe expansion a long time ago.
With this amount of PCIe, I don’t really find the challenge to use this as a workstation TBO. We do have insane USB cards that convert PCIe Gen3x8 into 8 USB3.2 10G ports. This would add cost to the system, but if you don’t need this amount of full-speed USB, then you would hava a lot of cheaper solution giving you 4 ports or so.
@emerth it exists. I have the Deep MATX version of this mobo and it has quite a lot of PCIE over MCIO cables as well as some PCIE x16 slots
What’s the idle power usage?
I love large dense boards like this! This is awesome. Rear I/O isn’t a big issue since there’s plenty of slots of I/O addon cards.
@Lance Longreen – can you post the full name & maker of your board?
Ahah – you mean this, right?
https://www.asrockrack.com/general/productdetail.asp?Model=ROMED4ID-2T#Specifications
Perhaps a stupid thing to ask…but with silicon photonics and all that jazz in vogue. I’m not sure why PCIE hasn’t shifted future development to a fiber optic for days + copper for power generation. I feel like manufacturing and tooling might make that fiscally impossible … but otherwise I’m confused.
All of these additional active redrivers snd complexity to keep signal integrity really makes you wonder about the future…
@emerth
I’m assuming it’s the ASRock Rack GENOAD8UD-2T/X550:
https://www.asrockrack.com/general/productdetail.asp?Model=GENOAD8UD-2T/X550#Specifications
@emerth did you check Asrock Rack homepage at all ? ;-)
Anyway here it is:
It has a mix of PCIE over the usual ×16 slots as well as MCIO connectors (Mini cooledge)
https://www.asrockrack.com/general/productdetail.asp?Model=GENOAD8UD-2T/X550#Specifications
@Nick, it is because normal ppl are not driving the market. PCIe5 exists because hyperscalers and such need it, not because normal ppl and businesses do. I think AMD has begun to accept this with some of it’s chipsets and mobo designs staying with reasonable-cost PCIe4.
@PatrickKennedy , Is there an chance of producing a full length pdf version of this review ?
For my reviews later when I goto think about possible upgrade paths for my present system .
When will the be a server motherboards with USB C (display port)?
Nothing wrong with VGA, just getting harder to buy monitors with VGA.
Never is the price mentioned…
@Gusty, when checking the webz for pricing I see pricing ranging from just over $1000 to close to $1200. So it will probably depend on reseller but it is likely somewhere in that price range.
is ther a optien to bifurcation on the slots?
I have this board with a 84c Genoa, its a great board, love the IPMI, its very extensive, as is the BIOS. My main problem is that I can’t install XCP-NG on it, the problem is the software, its refuses to find my NVME drive. Proxmox etc instals without a problem. The location of the NVMEs could be a bit better. This board is huge, it barely fit into my Meshify2XL. Would be nice if it had a USB3.2 header, but for the rest can’t complain