Lenovo ThinkSystem SD650 Water Cooled for Dense HPC Applications

The Lenovo ThinkSystem SD650 is certainly an interesting machine. As a 6U machine with four Intel Xeon Scalable CPUs per rack U it has a similar density to many 2U 4 node chassis. Unlike many of these designs, it is designed for higher wattage parts in higher density rack and aisle configurations by incorporating water cooling into the system.

Lenovo ThinkSystem SD650 Overview

One of the key aspects to the Lenovo Lenovo ThinkSystem SD650 is its six U design. Looking at the front of the chassis we can see features like a custom KVM connector, management and data LAN ports, USB ports and an expansion slot for Omni-Path and Mellanox FDR (and future) Infiniband.

Each U of the 6U chassis has on tray. Each tray holds 2 complete dual CPU systems for a total of four CPUs per U as well as some provision for expansion slots and storage.

Perhaps the first thing readers will notice is that there are no fans in the chassis which essentially removes all of the moving parts from a system, increasing reliability. Cooling instead is provided via a water cooling network that snakes through the two systems.

Lenovo provided a diagram showing how the cold water (blue) flows through the CPUs and memory as well as cooling the fabric cards and storage. RAM modules tend to use relatively little power, however NVRAM and Optane DIMMs use more power. While these are easy to cool with air cooling, if one removes fans from the mix, a cooling solution is required.

Using water cooling allows Lenovo to increase reliability as well as more efficiently cool servers. Using water cooling and heat exchangers can have massive cooling power savings, especially using free cooling heat exchangers rather than traditional chillers. More data centers are being built with water cooling facilities and HPC installations are certainly targeting water cooling for next-generation builds.

Storage is relatively minor. One can fit two 2.5″ SATA 7mm SSDs or a 2.5″ NVMe SSD in a standard 15mm form factor. One can also use m.2 SATA SSDs. In these systems, this storage is for tasks like holding OS images and some local data.

The rear of the chassis has six 1.3kW power supplies that also hold the systems fans. Not everything is water cooled. The rear also has the fan controllers and the water cooling manifolds that connect to facility water distribution sources.

Another key to this design is that it is intended for “future” processors with 245W TDP, not just the 205W TDP Intel Xeon Platinum 8180 that we have today. As TDP raises, and higher power devices invade RAM slots facilities have to deal with more heat in aisles.

Using free cooling and water-based heat exchangers is seen as a key technology to remove heat from higher power racks. Lenovo says that the 72 system rack with six of these 6U systems is expected to consume 43kW per rack.

Lenovo ThinkSystem SD650 Key Specs

Here are the key specs for the system: