AMD EPYC 7281 Linux Benchmarks and Review

An AMD EPYC 7281 CPU costs around $650 yet has a distinction: it provides about twice the cores that Intel provides in this segment of the market. With a total of 16 cores, 32 threads octo (8) channel DDR4 RAM and support for up to 2TB of memory, if you are looking to virtualize set of VMs with low CPU/ high RAM needs, this may just be your ticket. Aside from the massive CPU specs, the single socket configuration still has access for up to 128 high speed I/O lanes (e.g., PCIe.) What that practically means is that if you are running a dual Intel Xeon E5-2670 V1 server today, there is a good chance you can consolidate onto a single AMD EPYC 7281. Likewise, whereas we saw the single socket AMD EPYC 7251 as competitive with the Intel Xeon Silver 4108/ 4110, the AMD EPYC 7281 is where we can see AMD’s story of replacing dual socket Intel with a single socket AMD chip even of this generation.

AMD EPYC 7281 Overview

In our AMD EPYC 7281 in a dual socket review, we suggested that these chips are a significant performance upgrade over the AMD EPYC 7251. In a single socket scenario, that remains true however there is a credible question of whether the AMD EPYC 7301 or 7351P are worthwhile upgrades to someone who is set on the AMD EPYC 7281 regarding performance.

As a quick recap, the AMD EPYC 7251 sacrifices half the cores/ threads and supports only DDR4-2400 not DDR4-2666 memory at around $175 less. Saving 27% on just the CPU for a server with half the compute power is not generally something we advise so we like the AMD EPYC 7281 as a starting point. Moving up the stack, AMD has the EPYC 7301 at $825 and 7351P at $750. You can read about the AMD EPYC 7301 benchmarks and review where we show why the AMD EPYC 7351P is a better value for single-socket servers. Although today we are going to focus on the EPYC 7281, the AMD EPYC 7351P benchmarks and review shows why it is something special.

Key stats for the AMD EPYC 7281: 16 cores / 32 threads, 2.1GHz base and 2.7GHz turbo with a 32MB L3 cache. The CPU features a 155/170W TDP. Here is the AMD product page with the feature set. Here is the lscpu output for the processor:

The key here is that the single socket AMD EPYC 7281 uses a four NUMA node implementation per socket while Intel is currently using a single NUMA node per socket. You can read more about why in our AMD EPYC 7000 Series Architecture Overview for Non-CE or EE Majors article or learn about it in this video:

With this architecture, AMD is more sensitive to non-NUMA node aware workloads. On the plus side, with higher core counts AMD can maintain higher yields through a multi-die design.

Test Configuration

Although we are showing off platforms from other vendors, e.g., the Gigabyte platform in our AMD EPYC 7251 review, we now have every AMD EPYC SKU tested on a common Tyan EPYC platform. Here is the base hardware configuration we are using:

• CPU: AMD EPYC 7281
• Server Barebones: Tyan Transport SX TN70A-B8026 (B8026T70AE24HR)
• RAM: 8x 16GB 128GB DDR4-2666 RDIMMs (Samsung)
• SSD: 1x Intel DC S3710 400GB SATA SSD
• NIC: 1x Mellanox ConnectX-3 Pro EN VPI

Key to this system is that it supports 24x NVMe U.2 NVMe SSDs without using Broadcom PLX or Microsemi PCIe expanders. That is 96 lanes of PCIe 3.0 directly from a single SKU. One of the key advantages AMD EPYC has is that a single EPYC CPU can use 128x PCIe lanes, the same number as the dual socket configuration. Tyan has responded to this opportunity by offering a single-socket system that can handle 24x NVMe drives plus have I/O available for 10/25/40/50/100GbE.

AMD and Tyan originally suggested that we use a Samsung SSD (as pictured), however, to aid in consistency, we are using our lab standard Intel DC S3710 400GB SSDs.

In our forthcoming system review, we will have data on every CPU from the AMD EPYC 7251 to the EPYC 7601 for those looking at the system.

AMD EPYC 7281 Benchmarks

For this exercise, we are using our legacy Linux-Bench scripts which help us see cross-platform “least common denominator” results we have been using for years as well as several results from our updated Linux-Bench2 scripts. At this point, our benchmarking sessions take days to run, and we are generating well over a thousand data points. We are also running workloads for software companies that want to see how their software works on the latest hardware. As a result, this is a small sample of the data we are collecting and can share publicly. Our position is always that we are happy to provide some free data, but we also have services to let companies run their own workloads in our lab, such as with our DemoEval service. What we do provide is an extremely controlled environment where we know every step is exactly the same and each run is done in a real-world data center, not a test bench.

Python Linux 4.4.2 Kernel Compile Benchmark

This is one of the most requested benchmarks for STH over the past few years. The task was simple, we have a standard configuration file, the Linux 4.4.2 kernel from kernel.org, and make the standard auto-generated configuration utilizing every thread in the system. We are expressing results in terms of compiles per hour to make the results easier to read.

There are a few numbers we wanted to point out here. First, the single AMD EPYC 7281 is just between the dual socket Intel Xeon Silver 4108 and Silver 4110. It is fair to say we see a single AMD EPYC 7281 as equivalent to the dual eight core Xeon Silvers. Likewise, even if we move to the top of the Intel Xeon Silver range, the Silver 4116 at $1002, AMD still offers noticeably more performance in the $650 EPYC 7281.

c-ray 1.1 Performance

We have been using c-ray for our performance testing for years now. It is a ray tracing benchmark that is extremely popular to show differences in processors under multi-threaded workloads. We are going to use our new Linux-Bench2 8K render since it teases out more differences in this CPU segment than our older 4K results.

Here are our newer 8K results:

AMD always outperforms Intel in this type of benchmark. AMD relies upon Cinebench R15 heavily in their marketing. One item we can see is that the AMD EPYC 7281 is about equivalent to two AMD EPYC 7251 CPUs. The other stark contrast is seeing just how much faster the configuration is versus the single EPYC 7251. There are few $175 upgrades in a server that are as worthwhile as moving from an AMD EPYC 7251 to a 7281.

7-zip Compression Performance

7-zip is a widely used compression/ decompression program that works cross-platform. We started using the program during our early days with Windows testing. It is now part of Linux-Bench.

Here again, we see strong performance besting the dual Intel 8-core Xeon Silver CPUs and in a dead heat with the dual EPYC 7251 configuration.

NAMD Performance

NAMD is a molecular modeling benchmark developed by the Theoretical and Computational Biophysics Group in the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign. More information on the benchmark can be found here. We are augmenting this with GROMACS in the next-generation Linux-Bench, dubbed Linux-Bench2. With GROMACS we have been working hard to support Intel’s Skylake AVX-512 and AVX2 supporting AMD Zen architecture. Here are the comparison results for the legacy data set:

Without accessing AVX-512, the AMD EPYC 7281 performs well here. Although we have been focusing on the large delta between the 7251 and 7281, the additional clock speed and cache of the EPYC 7351P makes a strong case for a $100 upgrade.

Sysbench CPU test

Sysbench is another one of those widely used Linux benchmarks. We specifically are using the CPU test, not the OLTP test that we use for some storage testing.

Typically this is an excellent benchmark for AMD. We see that the AMD EPYC 7281 handily beats the Intel Xeon Silver 4116 which costs an additional ~50%.

OpenSSL Performance

OpenSSL is widely used to secure communications between servers. This is an important protocol in many server stacks. We first look at our sign tests:

Here are the verify results.

As part of the recurring theme, the EPYC 7281 is just between a dual Xeon Silver 4108 and 4110 configuration. We can again clearly see the benefit of the step up to the AMD EPYC 7351P despite having the same number of cores. We routinely get asked whether the additional cost fo the 7351P is worthwhile. This is another example of why we think it is.

UnixBench Dhrystone 2 and Whetstone Benchmarks

Some of the longest-running tests at STH are the venerable UnixBench 5.1.3 Dhrystone 2 and Whetstone results. They are certainly aging; however, we constantly get requests for them, and many angry notes when we leave them out. UnixBench is widely used, so we include it in this data set. Here are the Dhrystone 2 results:

Here are the whetstone results:

Overall, another set of solid results. We do see a close grouping between the 7281 and 7301 while the 7351P performs better. Given the EPYC 7351P is less expensive than the 7301, due to its single socket only capability, it is our recommendation for a step up in the stack.

GROMACS STH Small AVX2/ AVX-512 Enabled

We have a small GROMACS molecule simulation we previewed in the first AMD EPYC 7601 Linux benchmarks piece. In Linux-Bench2 we are using a “small” test for single and dual socket capable machines. Our medium test is more appropriate for higher-end dual and quad socket machines. Our GROMACS test will use the AVX-512 and AVX2 extensions if available.

When AVX-512 is enabled, the Intel Xeon Silver range fares considerably better. With that said, the Intel Xeon Silver range has modest AVX-512 capabilities compared to the Xeon Gold 6100 and Platinum 8100 ranges. We added a dual Intel Xeon Gold 6134 configuration (8 cores/ 16 threads each) to show just how much the dual port FMA AVX-512 matters.

Chess Benchmarking

Chess is an interesting use case since it has almost unlimited complexity. Over the years, we have received a number of requests to bring back chess benchmarking. We have been profiling systems and are ready to start sharing results:

Turning to our chess benchmarking results, we see a benefit from having fewer NUMA nodes. The AMD EPYC 7281 with four NUMA nodes out-performs the dual AMD EPYC 7251 configuration with eight. Although this benchmark is not overly sensitive, there is a benefit to consolidating from dual AMD EPYC 7251 to a single EPYC 7281.

A Note on Power Consumption

The other side of the equation is power consumption. The AMD EPYC 7301 is putting up some impressive benchmark numbers, but that does have an associated cost. Here is what we saw on our PDU after a few runs:

• Idle: 77W
• 70% Load: 204W
• 100% Load: 238W
• Peak: 301W

Note these results were taken using a 208V Schneider Electric / APC PDU at 17.4C and 71% RH. Our testing window shown here had a +/- 0.3C and +/- 2% RH variance. The figures are certainly more than the Intel Xeon Silver line, but with that extra power consumption, AMD is delivering a more expandable platform and more performance.

Market Positioning

Thes chips are not released in a vacuum. Instead, they have competition on both the Intel and AMD sides. When you purchase a server and select a CPU, it is essential to see the value of a platform versus its competitors.

1P AMD EPYC 7281 v. Intel Xeon Silver

The AMD EPYC 7281 is priced in the lower half of Intel Xeon Silver pricing ($650 in a range from $400-1000) yet has no direct Intel comparison regarding performance to the point it is often competitive with dual Intel Xeon Silver 4108 and Silver 4110 chips.

Intel’s strategy in the lower-end of the market is to position low power options only. All Intel Xeon Silver CPUs have a maximum TDP of 85W while the AMD EPYC 7281 is rated at 155/170W TDP. AMD essentially is providing the performance option with more cores, more PCIe lanes and more memory capacity to this side of the market. As we discussed in our AMD EPYC’s Extraordinarily Aggressive Single Socket Mainstream Pricing piece, AMD is simply providing more performance at a lower cost in this segment of the market with a game-changing pricing strategy.

1P AMD EPYC 7281 v. AMD EPYC

In terms of performance, the analysis is different when we look at the single socket AMD EPYC 7281 versus other AMD SKUs than when we looked at dual socket configurations. We see the biggest competition to the AMD EPYC 7281 in single socket configurations as the AMD EPYC 7351P. For about $100 more one gets a base clock boost of around 14% plus twice the L3 cache. In the context of a total system cost, that is likely under 4% more so we see it as a very worthwhile upgrade.

Concerning single socket value, we see the AMD EPYC 7281 as an almost must-do upgrade over the AMD EPYC 7251 as one gets an enormous performance boost, sometimes 2x, for around $175. When we saw the pricing and had the chips in the lab, the AMD EPYC 7281 offered a jaw-dropping value proposition.

Final Words

The AMD EPYC 7281 adds twice the cores of the AMD EPYC 7251 for a paltry $175 update. The move to price 16 cores / 32 threads and 8x DDR4-2666 channels with 2TB maximum RAM capacity in the heart of the Intel Xeon Silver range makes for an awkward comparison. From a pure performance perspective, Intel is not even competitive in this segment of the market. You need two Intel Xeon Silver 4108 or 4110 CPUs to get comparable performance to the single socket AMD EPYC. A single Intel Xeon Silver part will use less power which will impact TCO calculations; however, AMD has a great solution with the EPYC 7281.

At this point, we have published numbers for 8 of the 18 single and dual socket AMD EPYC configurations. We also have published numbers for Intel Xeon Platinum, Gold, Silver and Bronze including every Silver and Bronze CPU. We can confidently say AMD has an excellent value proposition here.