Intel has launched yet another performance-core Xeon line in 2024. This marks the third such platform in 2024 that can stake claim to being Intel’s top-end Xeon platform just in this calendar year. The 5th Gen Intel Xeon Scalable codenamed “Emerald Rapids” adds more cores, lots more cache, a handful of new features, and new underlying chip architecture all to a familiar platform. We have been working with the new chips for several weeks now, and have some thoughts both at the top-end, as well as the higher-volume market segments. As a bonus, we are going to give a preview into another deep-dive we will publish in a few days looking at just how fast the market is evolving for mainstream server buyers. It is another chip launch day at STH, so it is time to get into it.
5th Gen Intel Xeon Processors Emerald Rapids Resets Servers by Intel
For this, we have a video that also goes in-depth on the new processors. You can find that video here in the event you would prefer to listen.
Today we have the start of something crazy. At the start of 2023, the Intel 3rd Gen Xeon Ice Lake processors with PCIe Gen4, DDR4, and up to 40 cores were the top end of the market. On January 10, 2023, we had the 4th Gen Intel Xeon Scalable Sapphire Rapids launch which brought forth DDR5, PCIe Gen5, CXL 1.1 (Type 1 and Type 2), built-in acceleration, up to 60 cores, and more. Now, 348 days into 2023, we have a third generation of Xeon to claim the performance core crown, Emerald Rapids.
Let us be clear, one can think of Emerald Rapids (5th Gen) as an improved version of Sapphire Rapids (4th Gen) but they will work in the same platforms. The next big jumps are the new all E-core Sierra Forest line likely launching in June 2024, followed by the P-core Granite Rapids. We have already seen a number of motherboards for the 8-channel and 12-channel platforms coming in 2024.
For now though, the baseline has been the 4th Gen Xeon Scalable Sapphire Rapids that brought a host of new features. Let us just be a bit clearer, Sapphire Rapids was an enormous jump. Emerald Rapids is less so.
A big reason for this is that the platform is the same between them. With a BIOS upgrade, the same server can support 4th and 5th Gen Xeon CPUs. Since I did the launch videos for the current generation Supermicro X13 platforms here is a 1U system that we have now used with both generations of CPUs. More on what this is for later in this article.
The 5th Gen Intel Xeon CPUs now go up to 64 cores, but with a big caveat. The 64 core parts are less expensive (list price) because Intel has simplified the design. Beyond that the other headline feature should be the massively upgraded cache. We have chips with 48 cores and 300MB, 64 cores and 320MB, and so forth.
Memory is still 8-channel DDR5, but it is now DDR5-5600. UPI speeds are now up to 20GT/s for faster CPU-to-CPU communication.
One of the biggest changes is that Intel has changed its manufacturing process. Instead of four dies with four EMIB bridges, there are now two dies with up to 32 active cores each and three bridges.
That is for the XCC part. For the MCC parts, there is still a monolithic die. In one of the more challenging branding bits, we covered that there are EE LCC and EE MCC dies for the Intel Xeon EE with vRAN Boost SKUs which are now being branded as 5th Gen parts.
In Oregon, Intel gave us this nice little 32-core die trophy(?) to commemorate the launch.
Even without a macro lens, the Canon R5 with the 28-70mm f/2 still let us easily see the detail of the chip. The case was a dust magnet, and one can see how the core size compares with the dust strands on the case.
Intel (finally) supports CXL Type 3 devices with 5th Gen Xeon.
That means one can add four CXL Type 3 memory expansion devices, like this Astera Labs Leo board to a system. There are different modes of operation with interleaving and also treating memory as separate pools. At the same time, one can add a few CXL Type 3 memory expansion devices and get effectively around 12 channels of DDR5 memory bandwidth while also increasing capacity.
We have seen a number of demos of this and can confirm it is working.
Intel also has enhancements to its confidential computing with SGX and TDX with this generation.
Perhaps the biggest feature is that it can claim it has AI features built-in. This is not just AVX-512 VNNI like AMD has. Instead, this is AMX acceleration. As we saw in our Numenta Has the Secret to AI Inference on CPUs piece, both AVX-512 and AMX can be used on the CPU to increase the performance of AI inference.
In addition to having AMX, like the 4th Gen Xeon, the new 5th Gen has new turbo frequencies. These allow for better performance through maintaining higher clocks in most cases.
Intel has this slide with performance speed-ups. Please keep in mind, that the 3rd Gen Xeon Ice Lake SKUs topped out at 40 cores so there is a 60% increase in core count to 5th Gen. Likewise, there is around a 7% increase in core counts versus the 4th Gen Xeons.
Normally, at this point, we would delve into the new microarchitecture gains. We are not going to do that here. I asked Ronak Singhal an Intel Senior Fellow something along the lines of “From a core perspective, putting aside the L3 cache, this feels like there is not much difference. Is this more like (1st Gen) Skylake to (2nd Gen) Cascade Lake?” His paraphrased response was that the reason is that Cascade Lake had the fixes for Spectre/ Meltdown and VNNI so there was more in the Cascade Lake refresh. Instead, the work went into the manufacturing side.
That brings us to a pretty interesting result. At the same clock speed, workloads that do not use much L3 cache will end up performing similarly on Sapphire Rapids and Emerald Rapids.
Next, let us get to the SKUs.
