ASRock Rack EP2C612D16-2L2T Motherboard Review

The ASRock Rack EP2C612D16-2L2T motherboard being reviewed today is a harbinger of things to come. ASRock’s Rack division is dedicated to the server market. ASRock Rack has put together an Intel Xeon E5 V3 platform that many will find intriguing as it has both dual 1GbE and dual 10GbE links.

Test Configuration

We used a fairly typical DP setup for this type of server and what we have used previously in our Intel Xeon E5-2600 V3 coverage.

• CPU: 2x Intel Xeon E5-2690 V3
• Motherboard: ASRock Rack EP2C612D16-2L2T
• Memory: 8x SK. Hynix 16GB DDR4 2133MHz ECC RDIMMs
• SSD: 2x Samsung 840 Pro 256GB 4x Crucial M500 240GB, 3x SanDisk CloudSpeed 1000E 800GB SATA, 3x Intel S3700 100GB
• Operating Systems: Ubuntu 14.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5, CentOS 6.5

The test platform was rather expansive due to the enhanced SSD capacity of the motherboard.

ASRock Rack EP2C612D16-2L2T Overview

The ASRock Rack EP2C612D16-2L2T is a SSI EEB motherboard measuring 13″ x 12″. That is fairly standard these days. With the initial Intel Xeon E5 introduction, SSI EEB has become extremely popular as a form factor, especially with dual socket designs. The larger form factor is appropriate for most full-sized rackmount cases.

The motherboard is a dual socket, two DIMM per channel design meaning it has a total of 16 DDR4 DIMM slots. Using relatively inexpensive 16GB DDR4 RDIMMs one can easily hit 256GB RAM.

The CPU sockets use a narrow ILM layout. One needs to be aware of this when selecting heatsinks but options are generally available. The two CPU sockets are offset by more than 50%. Cooling of CPU2 will be impacted by heat generated by CPU1, however the offset should provide enough clean air in most chassis that cooling should be sufficient.

There are a total of six expansion slots, three in PCIe x16 physical slots and three in open-ended PCIe x8 slots. Unlike many competing motherboards, the slots have the same PCIe 3.0 electrical configuration as their physical configuration. That means there are 72 PCIe 3.0 lanes available via expansion slots which is excellent for this type of motherboard. One can add longer PCIe cards into the PCIe x8 slots because they are open ended and there is plenty of room behind the x8 slots.

Here is a quick screen grab. One can see that of the 80x PCIe 3.0 lanes, there are 72 anchoring the expansion slots and the remaining lanes are dedicated to the Intel X540 dual 10Gbase-T controller.

Between the Intel 612 PCH and and Marvell 9172 onboard SATA controller there are a total of 12 SATA III 6.0gbps ports. One can see in the above block diagram that the Marvell 9172 controller is connected to the PCH’s PCIe 2.0 lanes. For many 2U enclosures, the 12x SATA port are going to service the enclosures without requiring an add-in card. All 12 SATA ports are 7-pin designs which means simple point to point cabling. If one requires SFF-8087 connections for a backplane (as an example) it is easy enough to procure 4x 7-pin to SFF-8087 reverse break-out cables.

Behind the SATA ports is a USB 3.0 internal Type-A header. This is excellent for running an embedded image off of a USB key.

Next to the PCIe slots are an ASpeed AST2400 chip. We are starting to see the AST2400 become mainstream with the E5 V3 generation. During V1 and V2 generations, the AST2300 was more prevalent. Underneath the heatsink is an Intel X540 dual port 10Gbase-T controller. Generally the X540 runs hot so a heatsink is required.

Looking to the rear I/O panel, one can see a legacy serial and VGA connectors. There are two USB 3.0 connectors with an Ethernet port on top. This Ethernet port provides out of band management. The stack of two Ethernet ports are the 1GbE ports powered by dual Intel i210 controllers.

Next to the USB 3.0 ports are two RJ-45 connectors that are the 10Gbase-T ports powered by the Intel X540 controller. 10Gbase-T has become popular with the Intel Xeon E5 V3 generation as more and more next-generation applications are assuming 10gb connectivity and 10Gbase-T provides an easy migration path. One can use this motherboard as 4x 1GbE today then 2x 1GbE and 2x 10GbE as infrastructures are upgraded.

Overall, this is a very solid hardware platform for an Intel Xeon E5 V3 server.

Thermal Results

To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turn on FLIR’s MSX enhancements so we can see components outlined clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images.

Here we see thermal imaging results in-line with what we are accustomed to seeing from these platforms. 10Gbase-T requires more power than SFP+ based Ethernet due to increased signal processing requirements. One can see that even with our massive 4U coolers, the second CPU is slightly warmer. Overall acceptable results. One will want to ensure there is proper airflow over the X540 heatsink but in most enclosures this is not an issue.

ASRock Rack Management

ASRock Rack has an industry standard web management interface based on the American Megatrends MegaRAC solution. The ASRock Rack IPMI 2.0 implementation has an array of configuration options and sensor information. One of the more interesting features is the ability to graph sensor information over time.

ASRock Rack includes remote KVM-over-IP functionality. From what we understand, 80% or more of servers are now supporting IPMI 2.0 and some sort of KVM-over-IP solution. The reason is simple. In a datacenter, KVM-over-IP saves companies a lot of time by allowing remote administration instead of having to physically access the server. Companies like HP, Dell, Lenovo and Intel are charging more for full KVM-over-IP functionality, but ASRock includes it with the motherboard. That is by far the preferred way to bundle the functionality.

The ASRock Rack Remote management solution provides the remote administrator with the information they need to manage servers far away from home.

Conclusion

Overall one can tell that the ASRock Rack EP2C612D16-2L2T is part of a broader push into datacenters for ASRock. The company is pushing boundaries with including both dual Intel i210 gigabit Ethernet and dual X540 10Gbase-T Ethernet which provides capacity for both legacy 1GbE and newer 10GbE networks. The additional Marvell based SATA ports can help in situations where there are extra drives bays in a 2U chassis and one needs additional connectors. In terms of remote management, ASRock Rack makes an excellent product decision by bundling the KVM-over-IP functionality with the motherboard, differentiating over traditional server vendors.