Supermicro X10SRH-CLN4F Review – SAS3 HBA and 4x LAN

Our review of the Supermicro X10SRH-CLN4F is exciting. We review many motherboards each year and there are a few that provide an exceptional value due to their inclusion of high-quality onboard components. The Supermicro X10SRH-CLN4F is one of those motherboards. The short story is that it combines a single LGA2011-3 socket for the Intel Xeon E5 V3, a high-quality LSI SAS 3008 controller and quad gigabit LAN. This package is capable of replacing two Xeon 5500 series dual socket machines with less than half of the power consumption in a single node. Furthermore, it provides an excellent basis for storage servers. Price wise, the total package comes in at a lower price point than purchasing the LSI SAS 3008 controller and an extra Intel dual port NIC on their own. Let’s explore the motherboard.

Test Configuration

We used our standard UP setup for this type of server. We also used a higher-end dual socket capable Xeon E5-2600 V3 series chip instead of a Intel Xeon E5-1600 V3. This was to see if we could effectively consolidate two previous-generation dual socket setups into a single socket configuration.

• CPU: Intel Xeon E5-2698 V3 (16 cores/ 32 threads)
• Motherboard: Supermicro X10SRH-CLN4F
• Memory: 8x SK. Hynix 16GB DDR4 2133MHz ECC RDIMMs
• SSD: 2x Samsung 840 Pro 256GB 4x Crucial M500 240GB, 4x Samsung SM843 480GB, 4x Samsung SM1625 SAS SSD, 4x SanDisk Lightning lb206s SAS SSD
• Operating System: Ubuntu 14.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5

The goal of this configuration was simple: take 2x dual socket Intel Xeon L5520 generation servers (8 cores/ 16 threads each) and consolidate down to one single socket server. The total configuration idled at just under 50w (with only 2 SSDs) and peaked during Linux-Bench at 144w. This is essentially the same as one of the Xeon 5500 generation servers so one can now get similar compute density (albeit much faster with the E5-2696 V3) in half the power, half the footprint and one quarter the sockets as “power efficient” servers had in four generations ago. At $20/mo for 1A @ 120v and $30/ 1U that is $50/mo savings just on power and space.

We did use our standard add-in card suite including the Mellanox ConnectX-2 and ConnectX-3 cards, Intel 10 gigabit LAN cards, LSI SAS/ RAID controllers and etc. The Supermicro X10SRH-CLN4F has a built-in 12gbps SAS HBA and quad Ethernet making it highly capable on its own. Adding an Intel Fortville based dual QSFP+ 40GbE adapter turns this single socket server platform into an absolute monster.

Supermicro X10SRH-CLN4F Overview

The Supermicro X10SRH-CLN4F is a standard ATX size motherboard which means it will fit in almost any enclosure. The motherboard is laid out well for a number of applications. The 8x 4-pin PWM controlled fan headers found on the front and rear edges of the motherboard allow the motherboard to be placed in almost any chassis from low profile 1U chassis to large 4U cases such as the Supermicro SC846 series (24x 3.5″ bay 4U). That versatility makes the motherboard easy to work with in a variety of scenarios.

From an airflow perspective, the layout is extremely simple. The CPU socket uses a narrow ILM mounting point structure for thermal solutions. The socket is flanked on either side by 4x DIMM sockets for eight total. That means one can easily configure 64GB using lower cost 8GB DDR4 RDIMMs or up to 256GB using 32GB RDIMMs. Maximum memory configurations can hit 512GB using DDR4 LRDIMMs.

Delivering power we find a standard 24-pin ATX power connector and an 8-pin CPU power connector. This is a fairly typical configuration and easy to access given the location on the motherboard.

In terms of expansion slots, the X10SRH-CLN4F has six total. There is a PCIe 3.0 x4 electrical in an x8 physical slot which is great for common PCIe storage solutions. There are also three PCIe 3.0 x8 electrical slots. Two are in x8 physical and one is in an x16 physical. Beyond the PCIe 3.0 slots there are also two PCIe 2.0 x4 slots, one in an x4 physical and one in an x8 physical slot. Overall one has access to 28 PCIe 3.0 lanes and 8 PCIe 2.0 lanes. The Xeon E5 V3 platform  offers up to 40 PCIe lanes per processor but this motherboard has several onboard devices already using these ports.

One can also see in the above picture the PCIe based ASPEED AST2400 which is the BMC providing out-of-band management, VGA and other functionality to the motherboard. While some vendors are still using the older AST2300, it is nice to see Supermicro moving towards the more modern platform.

The next PCIe based add-on is the Avago-LSI SAS3008 controller onboard. This is LSI’s 12gbps 8-port SAS3 Host Bus Adapter that can also support RAID 0, 1 and 10 in software.  There are two connectors ready to accept SFF-8643 cables. This is the newer spec high-density SAS cable but it is easy to find SFF-8643 to SFF-8087 cables if you are using a chassis or backplane conforming to the older standard.

While the LSI SAS controller provides 8x 12gbps SAS ports and the ability to connect to expander backplanes, the Intel C612 PCH also provides 10x 6.0gbps SATA3 ports. These are standard on the platform but also mean that one can use up to 18x drives with the ports provided directly on the motherboard. This is perfect for 3.5″ 3U chassis. Adding another SAS HBA would give one access to 26 ports or enough for most 2U 2.5″ or 4U 3.5″ chassis. One can also use the onboard controller to pass-through to storage VMs such as an ESXi VM running a ZFS storage appliance while keeping high-speed SATA storage available for the datastore and other VMs.

In terms of internal USB headers, the two most interesting are the USB 3.0 front panel connector and USB 3.0 Type-A internal header. Many new cases are using this de-facto standard front panel connector which makes KVM cart data loads much faster and easier from cold aisles. The internal Type-A header is often used for embedded applications such as installing VMware ESXi hypervisor without requiring a hard disk.

The rear I/O panel has a standard serial console port, a 15-pin VGA port and two legacy USB 2.0 ports. These are fairly standard server ports over past generations. Complimenting the two USB 2.0 ports and internal USB 3.0 headers are two rear-panel USB 3.o ports. These are excellent for speeding up transfers from external devices (e.g. USB drives for OS loads.)

Above the USB 2.0 ports is a RJ-45 connector for IPMI/ and out-of-band management. This is a standard feature on newer Supermicro motherboards and is a great trend. Some vendors  require the purchase and installation of an add on module and/ or port to enable OOB management. While it only adds a few minutes upon initial installation configuration to not have this integrated, they take a considerable amount of time to install in the field for already deployed servers. Supermicro’s inclusion of the feature standard is an excellent choice other large vendors (Dell, HP, and etc.) should follow.

The quad LAN block supports four gigabit Ethernet links. These are controlled by the Intel i350-am4 controller. Some vendors have instead opted for the lower-cost quad i210 or 82574L setups, but using the premium Intel i350 quad port part is a premium component choice. The actual controller is covered by the black heatsink that can be seen above the VGA port. Generally the i350-am4 does not require a heatsink but adding one ensures that the gigabit Ethernet connections will remain stable.

Thermal Results

To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turned off the FLIR’s MSX enhancements so we can see hot spots clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images.

Here we see absolutely excellent performance from the Supermicro X10SRH-CLN4F. The heatsinks on both the SAS 3008 controller and the i350-am4 are certainly doing their jobs keeping those components cooler than we generally see in our motherboard thermal imaging results

Software and Management

Supermicro’s IPMI and KVM-over-IP as described a few times on this site, allows for a lot of deployment flexibility. Things such as fan speeds, chassis intrusion sensors, thermal sensors, and etc. can be monitored remotely. Alerts can be setup to notify the admins of issues. Beyond this, the functionality also allows for remote power control. You can do remote power up, power down, and reset of the server in the event that it becomes unresponsive. In fact, the test system has never had a keyboard, mouse, CD/DVD ROM, or monitor hooked up to it, even after multiple BIOS tweaks and operating system/ hypervisor installations.

Another important feature is the ability to remotely mount CD images and floppy images to the machine over the dedicated management Ethernet controller. This keeps maintenance traffic off of the primary Intel NICs. At the same time it removes the need for an optical disk to be connected to the Supermicro motherboard.

The latest revision of Supermicro’s management features include some key changes and new functionality. This includes the ability to recover from a bad BIOS flash from the IPMI interface.

We will have more on this feature soon, but suffice to say this makes management of remotely located servers significantly easier with some safeguard in the event a BIOS flash goes poorly.

Supermicro does include KVM-over-IP functionality with the motherboard. We have been testing servers from HP, Dell, Lenovo and Intel that all required an additional add-in key to get this functionality. It is an absolute time and money saver in the datacenter and other vendors should follow Supermicro’s lead in this space.

Conclusion

Overall the Supermicro X10SRH-CLN4F represents a tremendous value for those seeking a motherboard that supports quad Ethernet LAN and fast storage with the LSI SAS 3008 controller. Assuming a standard motherboard had a dual LAN controller standard, adding an Intel i350-t2 for the two extra LAN ports would cost around $130 from US retailers. Adding a LSI SAS 3008 HBA like the LSI SAS9300-8i would cost another $320. That is $450 in add-in equivalents on a motherboard that is retailing for $420 – $440. The motherboard also comes with the further potential to consolidate two older dual-socket machines into a single server which makes it an excellent value.