Intel Xeon Gold 6148 Benchmarks and Review The Go-To HPC Chip

When we pour over the semiannual Top500 supercomputer lists, a common sight in 2017/ 2018 has been the Intel Xeon Gold 6148. With a balance of 20 cores for multi-threaded performance and higher clock speeds than the Intel Xeon Gold 6138, it is a popular HPC compute platform. In our Intel Xeon Gold 6148 benchmarks and review, we will delve into what makes this chip so popular including the ability to upgrade to Intel Omni-Path 100Gbps fabric on each package for under $180, or about 15% of a PCIe Infiniband 100Gbps fabric card.

Key stats for the Intel Xeon Gold 6148: 20 cores / 40 threads, 2.4GHz base and 3.7GHz turbo with 27.5MB L3 cache. The CPU features a 150W TDP. This is a $3070 price point which is slightly lower than the price point of Intel’s previous generation Xeon E5-2698 V4 that this replaces. Here is the ARK page with the feature set. We also have a set of chips in the lab which are the Intel Xeon Gold 6148F parts. That F denotes on-package fabric, namely Intel Omni-Path 100Gbps. Along with the $174 price premium comes a 10W TDP bump for the chips but performance is the same as the non-F parts.

Here is what the lscpu output looks like for the F chips:

We are going to use standard Intel Xeon Gold 6148’s for this review, but the fact that Intel has the F versions of these chips may be intriguing to some. A word of caution STH found that Intel Skylake Omni-Path Fabric Does Not Work on Every Server and Motherboard.

This feature has made the Intel Xeon 6148F a popular HPC part since it can save several thousand dollars over a Mellanox EDR Infiniband solution.

Test Configuration

Here is our basic test configuration for dual-socket Xeon Scalable systems:

• CPU: 2x Intel Xeon Gold 6148
• System: Tyan Thunder CX GT75BB7102 1U
• RAM: 12x 16GB DDR4-2666 ECC RDIMM (Micron)
• NIC: Mellanox ConnectX-3 EN Pro 40GbE
• OS SSD: Intel DC S3700 400GB

The Tyan Thunder CX GT75BB7102 is a dual-socket server that crams a surprising amount of functionality into a 1U chassis. One can put a full complement of 24x DDR4 RDIMMs along with two PCIe x16 cards, multiple OCP mezzanine cards (storage and networking) as well as use M.2 SSDs.

Redundant PSUs flank each side of the motherboard, and Tyan uses a novel array of fans which combine to channel cold air over the parts of the server that need it most.

In terms of storage, along with the PCIe based options, there are 10x storage bays in the front. In our test system, we are using the Intel C621 PCH SATA lanes for a low power setup. One can also use SATA DOMs on the two standard 7-pin SATA connectors.

This configuration also comes with some nice features, for example, the RJ-45 networking ports on the back of the server are 10Gbase-T Ethernet.

Overall, this is a nice platform for low power servers.

The other important aspect is that we are keeping this test system stable and consistent for our benchmarking so we can have reliable power and performance numbers. Our readers often wonder, “which CPU should I get?” and that question often has three parts:

1. What is the cost?
2. What is the performance?
3. What is the impact to the price/ performance ratio of my applications?

Our goal with this series is to help our readers understand a relative performance ranking between CPUs. That way if you see two options in your configurator or from a sales rep quote, you have some idea of what you are getting with each option.