Welcome Back Intel Xeon 6900P Reasserts Intel Server Leadership

A Quick Word on Intel Xeon 6980P Power

We usually try to avoid testing power consumption in development platforms, but that is all we had.

Generally these platforms just blast fans 24×7 to keep parts cool, but that can add a huge amount of power. Also, this test platform had more of the temperament of my 4 month old son. Sometimes it worked flawlessly, and sometimes you uninstall four DIMMs and it freaked out.

What we will simply say is this, each CPU can use 500W. Add to that another 120W or more for 24 DDR5 DIMMs, and a few watts for storage and other board components, and you are at around 1.2kW without cooling. One notable omission in the new systems however is the PCH, so that is 10W+ and a discrete part no longer sitting on the motherboard. Still, adding 10-20% for fans depending on the form factor and 1.3-1.6kW is going to be the new range for a dual socket server, before add-in cards or storage are brought into the mix. If you add a 100W+ NVIDIA BlueField-3 DPU, those numbers go up.

Something that Intel did, which was great, is that it showed a look at the power consumption of the CPUs below 100%. Most server CPUs do not run at 100% load. In fact, for a huge number of workloads, you do not want to run a CPU at 100% because that is when you start to see higher tail latencies, packets processed slowly, and so forth. Most cloud instances run at very low CPU utilization (under 25%.) As a result, the actual load on most CPUs in virtualization clusters is well under 50% at any given time. With that context, Intel showed that in the 30-70% utilization range it can get up to a 1.9x performance per watt gain.

The chips may use more power per socket, but doubling the cores per socket while not doubling the TDP means we can get a lot more efficiency.

Key Lessons Learned: Market Impact

Normally we would go into a look at the impact to AMD’s parts, but let us take a step back for a moment. This is really the high-performance line from Intel. If you want better power efficiency for running web servers, Sierra Forest is the answer. If you want a storage server, you are probably buying an older generation CPU or waiting for the smaller socket P-core CPUs and the R1S configuration in Q1.

Since you can see it in the above photo, yes, there is an image of the upcoming 18A Clearwater Forest CPU where we expect the E-core CPUs to get very attractive.

Still, from a raw performance perspective the Intel Xeon 6900P with 128 cores is stellar and takes the performance crown for now. Of course, AMD said Turin is coming in 2H 2024, so we are not far from AMD’s modern offering. In any event, Intel will be competing with the same core count, and a more comparable process technology, instead of at a 50% or more core count deficit and much older process technology that it has found itself at for half a decade.

What is interesting is that competing with fewer cores meant that Intel has focused on things like AMX for AI inference acceleration, QuickAssist for crypto and compression that is now supported by default in Ubuntu.

For those looking at more inference, Intel is also launching its Gaudi 3 AI accelerators today.

The big change is that Intel is also saying it is good for NVIDIA head nodes.

In the AI industry, one of the reasons companies like Supermicro took off was offering AMD EPYC versions of NVIDIA compute servers when AMD EPYC Genoa offered many more cores than Intel. Still, when you look at very large HGX H100 deployments, the 5th Gen Intel Xeon Emerald Rapids was extremely popular even at a core count deficit. One of the big reasons was just Intel’s memory controller and I/O die configuration. That advantage should be maintained with Granite Rapids, and the AMD EPYC advantage for higher core counts will go away. Still, in most AI servers four expensive GPUs are sold for each CPU.

Final Words

Intel Xeon 6 introduces a lot more complexity. P-cores, E-cores, 8 channel or 12 channel memory configurations, different memory types, accelerators and so forth. Through all of that though, there is one simple message: Intel is back. New process and packaging technology is allowing Intel to get back to core-to-core competitiveness with AMD. AMD will fire its Turin salvo soon-ish, and we will be back to debating architectures and accelerators. No longer will the question be 64 EPYC cores versus 28 Xeon cores, or 96 EPYC cores versus 64 Xeon cores. Intel is back in the game.

Of course, Intel is pursuing a much more complex strategy splitting E-cores and P-cores. My feedback to Intel last week that Sierra Forest-AP needs to be pulled into Q4 2024 so that Intel can have its cloud native 288 E-core versus 192 core/ 384 thread debate this year. Those CPUs, however are for a different segment.

For now, Intel has finally solved its core count issue, while also offering things like AMX, QAT, and more for acceleration, high-end DDR5 and MRDIMM (MCR DIMM) solutions for big memory bandwidth, and more. Intel lost its top-end edge when it lost process leadership with 10nm and Ice Lake. As the manufacturing and packaging side has improved, Intel is back in the fight. Welcome back Intel.