For SC22, we had a number of new systems. Unfortunately, the numbers are a bit distorted again because stuffing the list has accelerated. As a result, we are releasing this on Thanksgiving a holiday associated in the US with stuffing. Let us get to the details and dig into some of the trends.
For those wanting to take a trip down into the archives, you can find our previous pieces here: June 2022, November 2021, June 2021, November 2020, June 2020, November 2019, June 2019.
Top500 New System Trends
First, we should highlight the sheer quantity of turnover in the Top500. When we started doing this analysis in 2018, we more than a quarter of the list turnover with each new publication. The pandemic in 2020 certainly had an impact, but we are largely on the other side of the pandemic in most of the world. This November 2022 list had new systems near peak pandemic impact levels of two years ago, and that was only so high due to stuffing:
There were only 41 new systems. June 2022 had only 39 systems, but that was also the list where a new #1 system debuted. With that, let us get into the trends of this small set of new systems.
Top500 New System CPU Architecture Trends
In this section, we simply look at CPU architecture trends by looking at what new systems enter the Top500 and the CPUs that they use. Let us start by looking at the vendor breakdown.
In June 2022, AMD edged out Intel for the most new systems on the list. Intel’s figures were significantly bolstered in this list from a Chinese HPC vendor’s list of stuffing practices using ancient CPUs.
When we go to CPU cores per socket, we see a few interesting tidbits. Notably, in the June 2022 list, the 64-core level had become the most popular. The eight-core systems are the NEC Vector Engine. We also had a single system with 16 cores (a frequency-optimized Milan part) in June 2022. Now we have a 12 core Cascade Lake system.
In the June 2022 list, the most popular architecture was AMD’s Zen3 or 3rd generation AMD EPYC. Here we can see a resurgence of 2019 generation Cascade Lake processors. This is due to the extreme stuffing that is taking place.
Here are the actual SKUs that are being used:
The overall set of new CPUs has grown to more unique SKUs with this list. Still, it is a disturbing trend to see Intel’s Cooper Lake Xeons not represented, Intel Ice Lake CPUs still very sparingly represented, and over 3-year-old Intel CPUs are the most prolific in the HPC world. On the AMD side, we see that the 2019 Rome generation is no longer represented. AMD has managed to move the HPC community completely to its 2021 products while Intel has been unable to move the community substantially off its 2019-era products.
This is again, due to stuffing by a Chinese HPC vendor, but in a world where we discuss replacement cycles of 3 years, it is worrisome to see this trend.
Accelerators or Just NVIDIA?
NVIDIA has been so dominant in the accelerator for HPC market that we have a section specifically addressing that in its title. In this list, it was nowhere near as dominant:
The AMD MI250X-based systems really made headway in June 2022 making up 7 of the 19 accelerated systems. In this list, it is only 2 of 13. While most of the Top 10 or Top 20 systems are accelerated, due to stuffing, in this list less than a third of systems use accelerators.
Unlike in CPUs, we are seeing 2022 generation NVIDIA H100 accelerators make a debut on this list in a single system.
Fabric and Networking Trends
Here is one that many regulars to this piece will identify with. On the Interconnect side, Ethernet has been by far the most common solution.
100GbE is at least a respectable interconnect. At the same time, we have seen a reversal of a trend. The 10GbE, 25GbE, and 40GbE systems had stopped appearing on the list. In November 2022, 10GbE and 25AGbE are back.
On the 11x 100GbE systems, those are all from Lenovo as we have come to expect:
Lenovo has three systems with HDR Infiniband, but 24 of its 27 systems use Ethernet, and 2 of 27 use relatively slow interconnects.
A few facts here:
- Aside of the Slingshot-based systems from HPE, no other vendor used commodity Ethernet on the list, it was only Lenovo.
- That 24 systems using Ethernet may sound like a familiar number with 24 new systems on the list also using relatively ancient Intel Cascade Lake Xeons. All of the Lenovo Ethernet systems are also using Cascade Lake Xeons.
Just taking a look at these systems, all without accelerators, we can see a few commonalities:
These are either software companies or service providers that have systems, with a few common core counts. This is a longstanding practice Lenovo uses to take portions of large installations, run Linpack, and submit results before the systems are used for more mundane tasks like serving web pages or building software.
Final Words
The Lenovo stuffing was not adding a system or two to its total. It is now the majority of new systems on the list with 24 of the new 41 systems being these Cascade Lake and Ethernet systems. It greatly helps Lenovo’s numbers and ability to say it has a leadership showing in the Top500.
Our readers are divided on their opinions of this stuffing. Some see it as fine. Others do not like this practice. What can be said is that Lenovo is stuffing the list with non-HPC systems. On the plus side, it is great to see that even though we have very new few Chinese systems on the list, at least we have Lenovo as a second China-based vendor participating as that makes the list more inclusive.
Another perspective to take on the situation is to wonder why there are so few new systems other than the stuffing systems. How many of the 120+ new systems appearing on each new list in past years were the type of stuffing that you are talking about? I can imagine it was easily 20 just as it was this past year, maybe more. Perhaps the difference isn’t an increase of stuffing but a paucity of legitimate new HPC systems. For example, why were there so few new AMD CPU and GPU systems in November after a relatively much stronger showing in June? Perhaps HPC centers are waiting for the next generation of hardware, eg Genoa, Sapphire Rapids, and H100, to be available. Or perhaps it has to do with budget considerations in the wake of exascale efforts, or budget considerations having little to do with exascale. Regardless, I doubt the answer to that has much at all to do with these Chinese stuffing systems. Haven’t they been an issue on the lists for at least the last 5 years?
The writing on this piece was a bit on the tiring side. Endless repetition on the stuffing part, bit no actual content. Who‘s doing the stuffing now?