Supermicro 2029UZ-TN20R25M Topology
Whoever made the Supermicro 2029UZ-TN20R25M system block diagram did a fantastic job. We wanted to call this out specifically because it is one of the best Xeon Scalable server block diagrams we have seen.
This diagram clearly shows the system’s topology. The Mellanox ConnectX-4 Lx dual 25GbE NIC uses lanes from CPU0. From this diagram, we can see another important aspect of this machine, one needs to have two CPUs installed for full connectivity. There are many systems where we see buyers purchase a single CPU configuration. This is a server that should only be purchased and used in a dual CPU configuration. So much of the server’s unique functionality is not utilized with a single CPU that we can make this recommendation.
One will also notice this is a high-performance 3x UPI link design. Some lower-cost models only have two UPI links between CPU sockets. That saves some power and motherboard cost at the expense of inter-socket performance.
Supermicro 2029UZ-TN20R25M Management
These days, out of band management is a standard feature on servers. Supermicro offers an industry-standard solution for traditional management, including a WebGUI. This is based on the ASPEED AST2500 solution, a leader in the BMC field. The company is also supporting the Redfish management standard.
On this server, we see similar features as we would across the Supermicro X11/H12 ranges. That means whether you are using an embedded Intel motherboard or a 4U EPYC storage server, you will have a similar look and feel to the management experience.
One can see one of the newer features in Supermicro’s management solution. Here there is NVMe drive status reporting.
Supermicro also has power monitoring features. This telemetry data is also available through the Redfish API which allows for more efficient fleet management and alerting.
In the latest generation of Supermicro IPMI is an HTML5 iKVM. One no longer needs to use a Java console to get remote KVM access to their server.
Currently, Supermicro allows users to utilize Serial-over-LAN, Java or HTML5 consoles from before a system is turned on, all the way into the OS. Other vendors such as HPE, Dell EMC, and Lenovo charge an additional license upgrade for this capability (among others with their higher license levels.) That is an extremely popular feature. One can also perform BIOS updates using the Web GUI but that feature does require a relatively low-cost license (around $20 street price.) That is a feature we wish Supermicro would include with their systems across product lines.
This pricing differential for a serviceable iKVM functionality is a big deal in this segment. Adding a $200 license to a server can add 10% or more in the embedded platform costs. Supermicro’s iKVM feature is extremely popular due to its inclusion with the server.
As a quick note here, this is one of the first systems we have that did not come with the default ADMIN/ ADMIN credentials that Supermicro has used for years. Instead, it is using the new randomized password standard. You can see the impact here.
What is more, you cannot set the ADMIN password to ADMIN on new Supermicro servers. Instead, the company is now requiring a minimum of 8 character passwords that meet specific criteria. This is going to be a significant change for many of our readers. Practically, it means one needs to log each individual BMC password while racking the server. Operationally, this is a big change.
Supermicro 2049P-TN8R Test Configuration
For this test we utilized the following setup:
- System: Supermicro 2029UZ-TN20R25M
- CPU: Intel Xeon Gold 6248R
- RAM: 24x 32GB DDR4-2933 ECC RDIMMs
- Storage: 10x Intel DC P4510 2TB, 10x Intel DC P3320 2TB, 2x Intel DC S3700 400GB (OS)
Overall, this is a simple configuration that is showing what we want to test. One of the more exciting aspects of this system is that we were able to test the new 2nd Gen Intel Xeon Scalable Refresh parts in this server.
Let us move onto performance before getting to power consumption and our final words.
It seems like an awful lot of storage bandwidth to the processor with very little bandwidth out to the network. Maybe database where the compute is local to the storage, but I don’t see the balance if serving the storage out.
It is a shame that Supermicro doesn’t offer that chassis/backplane in an single AMD EPYC CPU config, that combination of mixed 20 NVME + 4 SATA/SAS drives would be ideal while costing a whole lot less with a single EPYC CPU, while leaving plenty of available PCIe channels to support 100+Gbs network adapter(s) all without breaking a sweat.
Suggestion for the workload graph please: Dots and Dashes.
It’s tough to see what colour belongs with which processor graph line.
Thanks.
So just to confirm, 20 x PCI-e drives were connected to one of the few servers that can directly support this config. Then you install 20 NVME drives and didnt show that bandwidth or IOPS capabilities?
I apologize for dragging in Linus Sebastian of Linus Tech Tips fame because he is a bit of a buffoon, but he might have hit a genuine issue here:
https://www.youtube.com/watch?v=xWjOh0Ph8uM
Essentially, when all 24 NVME drives are under heavy load there seem to be IO problems cropping up. Apparently, even 24-core Epyc Rome chips cannot cope and he claims the problem is widespread – Google search results shown as proof.
I would like to hear from the more knowledgeable people that frequent STH. Any comments?
Good article, Patrick. However, with regards to the IPMI password, couldn’t it potentially be possible to change to the old ADMIN/ADMIN un/pw using ipmitool from the OS running on the server?
Philip – what about a bar chart?
Henry Hill – 100% true. With the whole lockdown, we had to use the Intel DC P3320 SSDs. Look for reviews on retail versions like we have. We have some of the only P3320’s in existence. They would not go overly well showing off the capabilities of this machine. Instead, we focused on a simpler to illustrate use case, crossing the PCIe Gen3 x16 backhaul limit.
Nickolay – check out the AMD/ Microsoft/ Quanta/ Broadcom/ Kioxia PCIe Gen4 SSD discussion. They were getting over 25GB/s using just 4 drives per Rome system. At these speeds, software and architecture becomes very important.
Oliver – there are new password requirements as well. More on this over the weekend.
Patrick,
A late catch-up, can you post a link to that AMD/… / Kioxia PCIe Gen4 SSD discussion, tks
Hi BinkyTo https://www.servethehome.com/kioxia-cd6-and-cm6-pcie-gen4-ssds-hit-6-7gbps-read-speeds-each/
What about storage class memory?
hi,
may i ask how it is possible to create a RAID5/ array with ESXi or Windows on bare metal?