At SC21 we were treated to something special. With CXL 1.1 generation chips like Intel’s upcoming Sapphire Rapids Xeons coming, the industry is hard at work enabling the new features and functionality. At SC21 we had the opportunity to look inside the Intel Eagle Stream development platform with the Emmitsburg chipset and Archer City RP motherboard to see what we may get in the next generation. Aside from the hardware, the demonstration of working CXL on a next-gen platform designed for developing CXL was probably the most important industry demo at SC21.
Intel Sapphire Rapids CXL with Emmitsburg Shown at SC21
The demo was jointly being shown at the CXL booth at SC21 by Astera Labs, Intel, and Synopsys. For those who are unfamiliar, Astera Labs is making CXL retimers, memory interfaces, and more. Synopsys is a major IP and validation player in the industry. Intel is… Intel.
On the front of the system, we can see a large fan array on top as well as 16x 2.5″ bays on the bottom. Again, this is a development platform, not something more meant for deployment like the Flex Bodega Bay Sapphire Rapids System shown at OCP Summit 2021.
For those who want to know what is on that label:
- Market Segment: Intel SDP for Xeon-SP
- Platform: Eagle Stream
- Processor: Sapphire Rapids ES1/B0
- Chipset: Emmitsburg
- Board Name: Archer City RP
- Phase: Alpha
Emmitsburg is the chipset or PCH. We started seeing it show up as a codename a few years ago, but this is perhaps the first official Intel showing that it is officially the PCH codename for Sapphire Rapids. This will replace the Lewisburg (LGA3647) and Lewisburg Refresh (3rd Gen/ LGA4189) generation of PCH’s for Intel’s server line.
As one can see, we have a dual-socket system that was actually running at the show.
The large black box someone told me was the Synopsys test box. For those wondering, yes it is sitting on foam suspending it from the motherboard. Again, this may have looked like a humble demo, but it was perhaps the most important for next-generation servers.
The view of the CPU and memory area confirms that we have 2 DIMMs Per Channel (2DPC) 8-channel memory. We can also see that the retention mechanism is similar to the current solution (we were told it is not the same.) If you want to get an idea of how this works, you can check out our piece: Installing a 3rd Generation Intel Xeon Scalable LGA4189 CPU and Cooler.
A few other points to note here. First, we can see a short copper heatsink below the black box and next to the foam. This should be covering the Emmitsburg PCH. The PCIe card we can see in the PCIe Gen5 x16 slot is there for the CXL demo that is being conducted here.
Only one of the 1.6kW PSUs was plugged in. One can also see various network ports, USB ports, and display outputs. On these types of development platforms, this I/O is often pretty rough but it is important since it helps bring up new systems.
As part of the demo, we can see the CXL link happening on port pxp2.pcieg5.port0 (CXL) here.
Also, there is an interesting tidbit since this was one of, if not, the first time the PCIe Gen5 system was shown off publicly. Here we can see:
- DMI is x8
- PXP1 is x16
- PXP2 is x16
- PXP3 is x16
- PXP4 is x4 — x4 — x4 — x4 —
- PXP5 is x4 — x4 — x4 — x4 —
- Port pxp0.pcieg4.dmi is x8
Using the first six lines we see the DMI x8 link for connecting to the Emmitsburg PCH. We also see five PCIe Gen5 ports, all with sixteen lanes. Three are x16. Two are bifurcated into four x4 links. Just for reference, here is this summer’s Architecture Day 2021 slide on the PCIe connectivity on the four compute die solution:
Of course, the big reason that this is exciting is that it is showing a 2022 server running CXL as part of the demo.
If you want to learn more about why CXL is important, check out Compute Express Link or CXL What it is and Examples.
I asked Raja Koduri during our Raja’s Chip Notes Lay Out Intel’s Path to Zettascale discussion and he said that CXL will become more important in the HPC realm around CXL 3.0. CXL 3.0 is not official yet, so it could be called CXL 2.5 or something, but everyone at the show was just calling it CXL 3.0.
Final Words
SC21 was much smaller, but it still had some gems. In this demo, we got to see some of the important work going on to enable CXL, a technology that will change the industry forever. We also got to see the Sapphire Rapids platform with the Emmitsburg PCH in action including the PCIe configuration as part of the Astera Labs, Synopsys, and Intel demo.
If you want to see a bit more via a video, this was actually featured in our Top 10 Showcases of SC21 video. There you may be able to see a few more angles of this awesome demo.
I don’t know what’s crazier —- STH was the only media to see this at SC or that it took you so long to show it other than in the video.
These don’t look like press photos…is there a reason we’re not getting larger versions even on click?
Good scoop regardless but I’d like to do some closer visual investigation.
Looks like a Synopsys HAPS-80 box, which means they may be emulating some of the functionality of the part still, or they have a “debug” package that allows them to monitor some of the internal path timing that wouldn’t be exposed in the production package.