Recently, Facebook introduced us to its next-gen Cooper Lake Xeon Platform. For those who are unaware Facebook is expected to be a major Cooper Lake customer (if not the largest) and driving force behind the platform. A few months ago, Intel confirmed first to STH that one of the two Cooper Lake platforms based on its next-gen Whitley platform is going away. Whitley was to be shared with the next-generation 10nm Ice Lake Xeons that will formally succeed the 2nd generation Intel Xeon Scalable processors we have today. Still, the Cedar Island platform is going ahead at Facebook and we have details of how Facebook will utilize the platforms.
Facebook Cooper Lake Generation Server Platforms Video
As with many of our articles this year, we have a video overview of this piece that you can find on the STH YouTube channel:
Feel free to listen to the video if you prefer to learn about these new servers that way.
Why Cooper Lake Matters
Cooper Lake is important for Intel as it is planned to publicly launch in the next month and a half. This is still a PCIe Gen3 14nm chip, but it has at least one killer feature: bfloat16.
This bfloat16 support is perhaps the killer feature of the new Cooper Lake platforms and is something that we expect to see more in future CPUs. Indeed, even the NVIDIA A100 just launched supports bfloat16. The new numerical format uses a sign bit plus a shorter 7-bit mantissa but the same 8-bit exponent as FP32 to provide similar accuracy for AI workloads while minimizing the bits that are moved around the system and computed. That gives both memory utilization benefits as well as about twice the performance as standard FP32.
Beyond this, the fact that Facebook is using the CPU is very important. Indeed, it is in some ways showing us why we are not seeing a Cascade Lake-D part in this generation when we look at the system design. It is also showing us what Facebook is using for mainstream and scale-up applications.
Mainstream 2-Socket Cooper Lake Platform
We have seen the mainstream 2-socket Cooper Lake Xeon Platform for some time. Our first public glimpse was from Facebook/ Intel at its OCP Summit 2019 Keynote.
That 2 socket platform provides a building block for larger systems we now have more information on. You can read more about some of the larger-scale systems in last year’s Facebook Zion Accelerator Platform for OAM.
This the internal view of Facebook’s next-generation 2-socket platform called “Sonora Pass”:
One of the strange bits from the OCP Summit 2020 is that it looks like Intel was trying to censor the fact that Cooper Lake Xeons in this generation have 6 memory channels. Looking underneath the air shroud, we can see that the two sockets each have 12 DIMMs.
Counting DIMM slots, we get a total of 12. Cooper Lake will support the new Barlow Pass Intel Optane DCPMMs which are the next-generation of DCPMMs after the current “Apache Pass” modules we have today. More on that when we discuss the Xeon D replacement use case.
When we zoom into the platform we can see:
- Two OCP NIC 3.0 slots
- Two PCIe FHHL slots
- One 3.5” SATA HDD slot
- Two USB 3.0 ports and OCP USB debug port
- Up to eight 40x40mm fans
- our E1.S 25mm for expansion
In the FHHL slots we can see M.2 accelerator modules. There is even a hard drive slot between the two PCIe slots. Boot options can include a SATA hard drive (via PCH) or M.2 SATA or NVMe boot devices. For storage, Facebook is adopting the E1.S 25mm so it is moving beyond just M.2 for storage here. Facebook’s decision to adopt E1 is also a likely reason we see E1.S announcements such as the Samsung PM9A3 this week.
A very interesting feature is that the platform is designed to be integrated into either 21″ or 19″ racks. You can see from the pictures that the motherboard does not reach across the entire chassis which helps enable that flexibility. A major step in OCP thinking is how to leverage designs across 21″ and traditional 19″ form factors. Higher volumes mean lower costs and increased quality. That is a driving force that will help OCP adoption.
Looking at the block diagram, we can see the SKT-P+ CPX Cooper Lake sockets tied via 10.4GT/s UPI links. That is the same UPI speed as current Cascade Lake Xeons which use Socket P.
We also see that we have PCIe Gen3 support even for the OCP 3.0 NIC slots. Counting lanes, it looks like these are 48x PCIe Gen3 designs as well. We also still have a PCH even as AMD and Arm competitors have moved to PCH-less designs in this market. The BMC is a standard AST2500/ AST2520.
Sonora Pass is a flexible platform so it seems as though we have a very good sense of the upcoming Cooper Lake platform’s key features.
While dual-socket servers are interesting, Facebook is using Cooper Lake somewhere else that may impact Intel Xeon D sales. We are going to explore that on the next page.
“but it has at least one killer feature: bfloat16” no, it’s not because Intel hasn’t GPU ready so they added it in CPU side; it’s already available in GPU : Radeon instinct and Nvidia
flo Weird way to look at it, Intel has other BF16 products available like Nervana & products from Habana Labs, why would they need a GPU for that type of workload? Makes no sense. If their customers want BF16 they are better of buying products that is optimized for BF16 workloads, a TPU style products is way more powerful than a GPU that only have a small part of it die dedicated to BF16 instructions, also, Instinct seems to only have BF16 available for ROCm(?)
And Intel expanding the BF16 options to more customers through AVX-512 on their CPU is not a bad thing. It has nothing to do with Intel not having a GPU ready. So yes, it is a killer feature to have it available on the CPU so it’s available to customers without them having to buy a compromised GPU or expensive ASIC/TPU/etc
At one point Intel was planning to fill a bunch of m.2 slots with NNP-I chips. Any word about FB still using those chips?
Are CPUs with avx512 bf16 vector processing more efficient than GPUs on some training operations using sparse datasets in FB’s Zion platform?
The rumors are that AMD is also adding AVX512 in zen4. I haven’t seen they are adding bf16, but seems like they’d want their cpus to be considered for FB servers.
This is a paper on the FB Zion training platform.
https://arxiv.org/pdf/2003.09518.pdf
Interested in knowing what options Facebook will have for accelerators in M.2 form factor. Also Facebook seems to be a diehard advocator for M.2 carrier of SSDs.
It seems weird to me comparing this to Skylake-D. Why would you need bfloat16 in a webserver, why would you want multiple NUMA nodes?
probably to do ML on the edge
They want to use the spare cycles of the web servers when it is the night in the region to help for the ML training.
“By skipping a “Cascade Lake-D” solution, Facebook can get Barlow Pass DCPMMs that run at DDR4-2933 speeds. One can see how this solution helps unify Facebook’s deployment stack.”
There is speculation that Barlow Pass (gen 2 DCPMM) doubles gen 1 max capacity to 1TB per stick … also that it can go do 3200MT/sec. Perhaps this memory capacity feature is becoming more attractive as models keep growing.
https://www.theregister.co.uk/2019/09/26/intel_gen_2_optane/
is that single NIC doing virtual functions to support the networking of many nodes behind it?
“Cascade Lake will support the new Barlow Pass Intel Optane DCPMMs which are the next-generation of DCPMMs …”
Should be “Cooper Lake will support …”