Recently a new Supermicro Hyper-Speed server was sent to STH for review. For those not familiar with Supermicro Hyper-Speed it is enterprise overclocking. Basically, Supermicro enables mild overclocks on their Hyper-Speed products. These overclocks are innovative in the data center because dual socket Intel Xeon E5-2600 series CPUs are multiplier locked. The Supermicro Hyper-Speed solution can handle 150w Intel Xeon E5-2687W processors and overclock them as well as overclocked DIMMs. The key here is that Supermicro picks parts and overclocks. It does so to validate that the systems will work in data center environments in a reliable fashion.
Test Configuration
Supermicro sent the following test configuration for our testing. This represents one common configuration for a compute node. One other popular configuration is using dual Intel Xeon E5-2637 CPUs (4C/8T) for applications where one needs high clock speed and lower core counts due to per-core license costs.
- Intel Xeon E5-2687W @ 3.224GHz base clocks
- Supermicro SYS-6027AX-TRF
- Supermicro X9DAX-iF Motherboard
- Samsung 8GB x 8 DDR3-1600 running at 1937MHz
Overall, this is the fastest dual socket configuration available in terms of today’s processors. Currently the Intel Xeon E5-2600 series line tops out with Intel Xeon E5-2687W at 3.1GHz base clocks and 8 cores / 16 threads each. As we will see, overclocking the CPU by 4% and memory by 21% yields the fastest configuration we can get at the moment.
Supermicro Hyper-Speed Server Internal Components
Inside our Supermicro SYS-6027AX-TRF is a X9DAX-iF motherboard with the dual Xeon E5-2600 series overclocked CPUs, memory and various other components. Looking under the top shroud we can see the absolutely massive heatsinks the system employs to keep everything cool and reliable. The Supermicro X9DAX-iF is a 13.68″ x 13″ EE ATX motherboard. After seeing this in person, I would highly recommend only using this with the Hyper-Speed server and workstation chassis designs. The spacing in the chassis does appear to be specifically designed for these large motherboards. For those wondering, the sparse space on the Supermicro X9DAX-iF flanked by the CPU heatsinks and the power supply is populated with a LSI SAS 2208 controller on the X9DAX-7F and X9DAX-7TF. The latter also adds 10GbE support to the platform with the 7 denoting the LSI SAS controller. When one orders Hyper-Speed systems from Supermicro or its partners, these are options that share the same platform seen below.
Looking at the heatsinks from another angle, one can see that they fill the 2U chassis above the LGA 2011 sockets, and across the memory slots.
Here is a close-up of one of the Supermicro SNK-P2048P heatsinks. One can see the large heatpipes that move air through the assembly. One can also see the extent to which the heatsinks overhang the memory. Suffice to say, Supermicro is channeling air effectively here over both the processor heatsinks as well as the memory.
Supermicro validates the memory for overclocking also. The CPU heatsink needs to be removed in order to service the DDR3 DIMM slots, so it is important that these are reliable components.
One can see that Supermicro uses Samsung DIMMs and they are on the Hyper-Speed compatibility list. As we will see in the next article, these registered ECC DDR3 DIMMs are specified to run at 1600MHz but Supermicro Hyper-Speed runs them at 1937MHz for a 21% overclock. The Supermicro Hyper-Speed solution supports up to 512GB of Registered ECC DDR3 in 16 DIMM slots.
Feeding the hot swap backplane is the familiar C602 chipset courtesy of the Supermicro X9DAX-iF motherboard.
For expansion slots, the Supermicro SYS-6027AX-TRF has two PCIe x16 low profile slots that are situated away from the CPU heatsinks. The blue slot is a Supermicro proprietary UIO slot. UIO is more or less PCIe x8 reversed so the components are on the other side of the board, a trait useful in low-profile systems. Also, UIO cards tend to be priced lower than their PCIe counterparts. When one purchases a SYS-6027AX-TRF adding these UIO options can save up front purchase costs. If one does not use the UIO slot, the PCIe slot is x4. The other three PCIe x8 physical slots are also there for more expansion capabilities. Overall, one can add a lot of hardware into one of these machines. With redundant 1.28kW power supplies, power is not an issue even if the slots are completely full.
Closing Thoughts
The test Supermicro SYS-6027AX-TRF system utilizes a X9DAX-iF motherboard. Supermicro offers SKUs with -7 denoting LSI SAS 2208 onboard, -T denoting Intel X540 dual 10GbE and a SKU with both. Overall, there are a lot of configuration options available with the Supermicro Hyper-Speed platform if one wants to take advantage of local I/O along with the overclocked processors and memory. Again, given the platform considerations, I would strongly advise purchasing the Supermicro Hyper-Speed platform with the SYS level parts (system) or as pre-configured systems. Some not familiar with Supermicro may know that the company sells components. As the company has grown, it is putting a greater emphasis on fully pre-configured systems. This is one platform so well integrated, that I would recommend purchasing the entire Supermicro Hyper-Speed solution so one can have a validated overclock that runs reliably.
Big heatsinks there! You mention LSI SAS 2208 and X540 10GbE options. Are there onboard FDR Infiniband options also or do you need to use an expansion slot?
The layout makes sense. Focus is on cooling the CPUs and memory.
Also interested in dual port onboard FDR. I guess using expansion cards makes sense too.
Would love to see more on the performance side.
Looking forward to seeing your STREAM benchmark results on that overclocked memory!
Actually, linpack is responding a bit to the overclocked CPU and memory also!
I think if you are going to test this system, the question is, does the overclocking introduce errors and how often? What would be a great value-add test is to run something like Folding@Home for 2 weeks, maybe a month and report how many times it crashed due to the overclock. That would tell people thinking of using this for scientific calculations what to expect.
Also, I noticed that the overclock with 16 GB Dimms is pretty low, you have to use 8 GB Dimms for more extensive overclocking.