Supermicro SYS-240P-TNRT 4P 3rd Gen Xeon Scalable Server Hands-on

Hot on the heels of today’s 3rd Generation Intel Xeon Scalable launch for the platform codenamed “Cooper Lake” or “Cedar Island” we have the Supermicro SYS-240P-TNRT. The SYS-240P-TNRT is a 4-socket server designed to leverage the new capabilities of Intel’s new Xeon platform. As shelter-in-place restrictions in the Silicon Valley eased, I was able to check one of these servers out at Supermicro headquarters.

Video Version

As with many of our pieces this year, this one has an accompanying video.

We are going to have more detail in this article, but want to provide the option to listen. As a quick note, Supermicro filmed the video bits at their HQ, provided the systems in their demo room, and their product managers that were able to make it. We did a whole series but are tagging this as sponsored since we relied upon their facilities instead of our own. I was able to pick the products we would look at and have editorial control of the pieces (nobody is reviewing these pieces outside of STH before they go live either.) In full transparency, this was the only way to get something like this done, including looking at a number of products in one shot, without going to a trade show during shelter-in-place. Look for more in this series coming to STH over the coming weeks.

Supermicro SYS-240P-TRNT 4P 3rd Gen Xeon Scalable Server

The Supermicro SYS-240P-TNRT from the front view looks like a typical 2U server from the company. As a fun note, we swapped in alternating drive trays to remind me that the system can support either 2.5″ NVMe (PCIe Gen3) or 2.5″ SATA/ SAS drives for the front storage.

Just behind the front storage are four powerful fans. Some other vendors are using more than a dozen fans to cool the system. Supermicro is using four plus two power supply fans. That cooling is important since a big part of the new 3rd Generation Intel Xeon Scalable platform is higher TDPs.

As a quick bit of background, the 3rd generation Intel Xeon Scalable processor family codenamed “Cooper Lake” adds a number of upgrades over the previous generation Cascade Lake. One of those, as previously mentioned is the ability for the platform to support 250W TDP SKUs such as the Xeon Platinum 8380H and Platinum 8380HL which are the top-bin 28-core parts for the server. As a fair warning, Supermicro did not let us get our hands on the machine with CPUs. Sill, we got to see the system in-person.

While there are core enhancements such as bfloat16 and some security and RAS improvements, the Cooper Lake Xeon generation is designed primarily to be a next-gen quad-socket platform. One of the biggest features is the 6x UPI links per CPU allowing for the same dual 10.4GT/s UPI links between Cooper Lake processors in 4-socket as the 2nd Gen Intel Xeon Scalable Refresh SKUs have in 2-socket configurations. Effectively, Cooper Lake erases the inter-socket bottleneck normally associated with the previous generation of 4P Xeon designs.

As you can see, the sockets are new as well. The SYS-240P-TNRT uses four Socket P+ sockets. These are new 4189-pin sockets with a new mounting mechanism for cooling solutions that is considerably more robust.

Each socket has a 6-channel memory controller. DDR4 support is up to DDR4-3200 for the Platinum 8380H up from DDR4-2933 on the Platinum 8280 generation. One can use two DIMMs per channel which means that the Supermicro SYS-240P-TNRT can support up to 48 DIMMs total. Of those DIMMs, half can be the new Intel Optane PMem 200 modules. These are the next generation Intel Optane DCPMMs with a new controller that provides higher bandwidth and lower power operation.

Using a 256GB DDR4 module and a 512GB Optane PMem in each of the memory channels of the Supermicro platform yields 4.5TB per socket or 18TB total.

Moving to the rear of the chassis, we see this entire server is powered by two 80Plus Titanium level power supplies. A change for Supermicro in this generation is that each power supply is located on opposite sides of the chassis. Along with that, the platform can offer networking and risers on each side of the system that is virtually symmetrical. This is to allow data center operators to setup A and B sides that are completely independent for power and networking. It is a small detail, but an important one on this system since it is different than previous generation Supermicro designs.

One will notice that the system has support for multiple PCIe cards which can be for storage, accelerators, or other I/O. In this generation, there is even an OCP NIC 3.0 compatible module slot which can handle NICs like the upcoming E810 Columbiaville NIC we saw at OCP Summit 2019. With this generation, the Intel “Lewisburg” C620A chipsets do not have onboard 10GbE capabilities, so having swappable NIC modules is important.

If you look closely, you will notice that in the expanded view of a full accelerator configuration there are PCIe risers above the PSUs. In the system at Supermicro HQ, there were no PCIe riser slots for that. Perhaps one of the most exciting developments is that this Supermicro X12 platform is relying heavily on cabling to provide flexible risers. This allows PCIe lanes to be routed to different expansion cards and NVMe SSD locations. It also allows for a very flexible model to support different PCIe I/O (e.g. on one server a location can have an x8 slot and the other it can have an x16 slot.

This cabled riser functionality is something we are going to see more of in the industry as we move to future PCIe Gen4 and Gen5/ CXL standards. Cabling allows for more flexible configurations and longer distances to be spanned. While some may see a bunch of silver connectors, STH readers should see the future direction of servers.

Final Words

Overall, the Supermicro SYS-240P-TNRT is a very different server than I expected. I thought that it would be a platform more akin to either the Supermicro 2049P-TN8R or Supermicro SYS-2049U-TR4 we reviewed based on 2nd Generation Xeon Scalable processors. Instead, Supermicro is taking this opportunity to make an enormous number of changes to how they are designing systems in their X12 generation for 3rd Gen Xeon Scalable / Cooper Lake servers. It is great to see Supermicro offer a new solution and then innovate on many of its underlying technologies at the same time.

Hopefully, you found this piece interesting. We have several more of these coming from my visit to Supermicro since it is unlikely we will see these systems at trade shows any time soon.