Supermicro X9DRH-7TF Motherboard Review – This board ROCKS!

Today we are looking at the Supermicro X9DRH-7TF which is one of our highest-rated products on the site. Simply put, this is one of those boards that, if it fits into your architecture, is a platform for a very high-end server. Here is the quick spec sheet: two LGA2011 sockets each with 8 DIMM slots. Onboard 10 GbaseT dual port Ethernet. Onboard LSI SAS2208 controller with 1GB of cache. If that is not enough, the motherboard also has seven PCIe slots. About 15 months ago Pieter previewed the Supermicro X9DRH-7TF calling it the “ultimate” server motherboard. After having one running in the lab for about a month, I can concur so long as this is what your infrastructure needs.

Test Configuration

The initial thought was to just replace the motherboard in our standard LGA2011 test bed, but we decided to skip that in favor of building a more appropriate test bed for this motherboard. We utilized the fastest LGA2011 “server” processors available right now (we decided against the E5-2687W’s we have because those are ostensibly for workstations.) We loaded up a healthy dose of RAM and more SSDs to the mix.

1. CPUs: 2x Intel Xeon E5-2690 CPUs
2. Motherboard: Supermicro X9DRH-7TF
3. Memory: 16x Samsung 16GB 1600MHz DDR3 ECC RDIMMs
   
4. SSDs: 2x Samsung 840 Pro 256GB 4x SanDisk Extreme II 240GB
5. Operating System: Ubuntu 12.04 LTS and Windows Server 2012

Overall this ended up being a nice test platform and is certainly a fairly high-end server for those companies that do not have access to Ivy Bridge-EP based chips or Haswell-EP chips yet.

Supermicro X9DRH-7TF Review

The Supermicro X9DRH-7TF has a lot of characters. As we see the 7 in the name we know there is a newer generation 6.0gbps LSI SAS controller onboard. The T tells us that we have 10GbE. Supermicro rounds it out with IPMI 2.0 onboard. Overall, some very high end components onboard that we will get to soon.

One thing you will quickly notice in the overview shot is that the motherboard utilizes the narrow ILM for LGA 2011. We did a piece on square versus narrow ILM which is certainly an important read for those not familiar. This helps cram a lot of functionality into the EATX form factor.

Supermicro provides a nice diagram with all of the major components. This motherboard is packed with features so it serves as a map of all of the connectors one would typically look for, and some of the onboard controllers.

Since there are two big-ticket onboard devices, the block diagram becomes important here. This is because if only one CPU is installed, the motherboard cannot be fully utilized in terms of expansion slots. This is certainly a platform for dual processors. As one can see, both the Intel X540 network controller and LSI SAS2208 SAS controller are wired to CPU1. This means that one will have the onboard components active with just one CPU. It also means that in a single-CPU configuration one can only utilize three of the six PCIe 3.0 x8 slots and not the single x16 slot.

Looking at the bottom corner of the board, we find the rightmost heatsink covering the LSI SAS2208 controller. Underneath the controller are the DRAM cache for the controller. Connectivity is provided by two SFF-8087 connectors at the edge of the motherboard. Suffice to say that this is a major factor.

The implementation of the LSI SAS2208 generally comes in a PCIe 2.0 or PCIe 3.0 generation. One can see this in the LSI cards, the 9266-8i and 9271-8i. Both cards use the SAS2208 controller, with the latter being the PCIe 3.0 variant. Like the LSI 9271-8i Supermicro implements 1GB of DDR3 and one can add a cache protection device to keep data safe during power outage. For those wondering, the comparable add-in card, the LSI 9271-8i sells for around $600 alone. We will have a breakdown later, but integrating onboard components saves a significant amount of money with this platform.

Also of note, Supermicro gives access to 8x SATA II 3.0gbps ports and 2x SATA III 6.0gbps ports from the Intel 602 chipset. Without installing an add-in card this gives the motherboard access to up to 18 connected drives.

If one did want to install expansion slots, the motherboard has six PCIe 3.0 x8 slots. In addition there is a longer PCIe 3.0 x16 slot. This means that one can add an additional 7x PCIe devices to the platform. One thing that can be seen clearly from this view is that the PCIe 3.0 x8 slots are all open ended. One can install x16 PCIe devices in the slots and have them run in x8 mode.

The picture below shows a different angle on the PCIe slots. One can see that Supermicro’s engineers made the area behind the PCIe slots relatively clear so one can add larger cards easily. As a side note, I give the Supermicro folks feedback that I strongly prefer open ended slots just because it allows more flexibility by removing the small piece of plastic on the x8 slot. Great to see this is taken into account.

The other major feature in the picture below is a heatsink. The Intel X540 10 gigabit controller runs cooler and uses less power than its predecessors, but it still needs a heatsink and airflow. This is the same controller one will find on the Intel X540-T2 network adapter, a device that costs approximately $550.

Moving to the rear I/O, we can see the two RJ-45 connectors for the 10 gigabit base-T Ethernet. The one big advantage of base-T connectors is that they are very easy to integrate into an existing 10/100/1000 infrastructure. Some users may prefer SFP+ connectors so they can run fiber, so a prospective buyer should simply ensure they are purchasing a motherboard suited to their 10GbE infrastructure.

Other rear I/O panel features are a standard serial port, 15-pin VGA port. One will notice the combo PS/2 connector found on older generation motherboards is absent. This is certainly a trend with newer generation motherboards and it is about time that after well over a decade with USB and three major revisions already, we are seeing USB take the lead. One will also see a RJ-45 connector stacked above the USB ports. That is a RTL8201N based port that allows for out of band IPMI 2.0 management and monitoring.

Supermicro Remote Monitoring and Management

Supermicro’s IPMI and KVM-over-IP as described a few times on this site, allows for a lot of deployment flexibility. Things such as fan speeds, chassis intrusion sensors, thermal sensors, and etc. can be monitored remotely. Alerts can be setup to notify the admins of issues. Beyond this, the Nuvoton BMC chip also allows for remote power control. You can do remote power up, power down, and reset of the server in the event that it becomes unresponsive. In fact, the test system has never had a keyboard, mouse, CD/DVD ROM, or monitor hooked up to it, even after multiple BIOS tweaks and operating system/ hypervisor installations.

Another important feature is the ability to remotely mount CD images and floppy images to the machine over the dedicated management Ethernet controller. This keeps maintenance traffic off of the primary Intel NICs. At the same time it removes the need for an optical disk to be connected to the Supermicro motherboard. Combined with the onboard USB 2.0 header, mounting OS installation or recovery images remotely is a very simple affair.

In combination with the baseboard level management features just described, KVM-over-IP features of the Supermicro board are not to be underestimated. One can log on to the server using a web browser with Java platform support to administer the server. Or one can go through Supermicro’s IPMIview software and have remote console capabilities, with mouse support included. Oftentimes, users opine that shell access is enough to troubleshoot. However this BMC level access to the remote system allows one to immediately see things such as a hung and incomplete boot process, frozen servers, as well as manage motherboard and add-in card BIOS remotely. Supermicro provides a well featured WebGUI to its IPMI management system for each server.

IPMI 2.0 features, including the KVM-over-IP features are essential for a server-class motherboard unless one has an external KVM-over-IP unit and a network addressable power source. For users with redundant PSUs, as will be the most common use case with this motherboard, remotely toggling on/ off multiple power supply ports across multiple PDUs can be difficult making the motherboard option preferred.

Conclusion

Overall, the Supermicro X9DRH-7TF is a very strong value. While that may seem strange for a motherboard that retails for just over $700, one needs to look at the onboard components involved. As mentioned earlier in this review, there is essentially a $600 LSI SAS RAID controller onboard and a $550 10GBase-T network controller embedded in the motherboard. That is $1150 of integrated components. Add-in another $350 or so for a similarly expandable dual socket LGA2011 motherboard with IPMI and one can quickly see that this is a $700 package that integrates three major system components that purchased separately would retail for approximately $1500. The big considerations here are twofold. First, one needs to ensure that the onboard components are required for their infrastructure. If one needs neither the X540 Base-T or SAS2208, then there are other motherboards better suited to the job such as the X9DR7-LN4F we use for CPU benchmarking (integrated Intel i350-t4 and SAS2308 HBA.) The second aspect is that integrated components are not portable. Unlike with their add-on card components, integrated components stay with the motherboard/ platform. There is a positive side of this though. Physically, 9x PCIe slots are hard to fit onto a standard sized motherboard (excluding form factors such as SWATX for the time being.) By integrating components, one can use a total of 7 add-in cards, essentially avoiding two for the dual port 10GBase-T and SAS2208 controllers.

Bottom line, the Supermicro X9DRH-7TF is a package that includes top of the line components at what amounts to a 53% discount over buying them separately. If this fits into your overall IT architecture it makes an extremely strong case as a high-performance and high-value platform.