ASRock Rack EPC612D4I Review – a mITX Xeon E5 platform

When we first saw the preview press release of the ASRock Rack EPC612D4I needless to say, we were excited. We covered the launch here and gave a sneak peak at the motherboard with its new DDR4 ECC SODIMMs from Innodisk. The quick summary is that the ASRock Rack EPC612D4I is a mITX platform that supports four SODIMM slots. When the board arrived we knew we had something special. Just to see how the unit fared, we put the setup into our Fremont, California colocation facility to see if such a small package could deliver in our rack. The platform has now had about a month of uptime without a single hiccup.

Test Configuration

We only had two 8GB ECC DDR4 SODIMM modules to test with as none were available publicly when we received the unit and Innodisk did send us two sample modules which helped tremendously. In terms of processor configurations we had two choices: a low end low power CPU or a higher-end multi-core CPU. We quickly decided against trying to cool an E5-2699 V3 or E5-2698 V3 in a tight mITX motherboard in a 1U chassis. Instead we opted for a 120w TDP processor to lighten the load.

• CPU: Intel Xeon E5-2695 V3 (14C / 35MB LLC)
• Motherboard: ASRock Rack EPC612D4I
• RAM: 2x Innodisk 8GB ECC DDR4 SODIMM
• SSD: 2x Samsung 845DC Pro 400GB
• Chassis: Supermicro
• OS: Windows Server 2012 R2, Ubuntu 14.04.2 LTS, Proxmox VE 3.4

Overall, we would (really) have liked to use more RAM than just over 1GB/ core but the setup did work well for our needs.

ASRock Rack EPC612D4I Overview

The ASRock Rack EPC612D4I is a compact mITX package meaning it measures 6.7″ x 6.7″ for 44.89 square inches. When you consider that a LGA2011 narrow ILM dimensions are ~8.16 square inches over 18% of the surface of the board is basically taken up by the CPU. Significantly more if you include the extra latch dimensions for the socket.

One can see the four SODIMM slots, power connectors, USB header, PCIe x16 slot and SATA connectors are all aligned to aid airflow. The prospect of adding some of the larger processors to such a small form factor does tend to make one think conservatively when planning for airflow.

The PCIe 3.0 x16 slot is located in the standard position at the bottom of the motherboard. Just above the PCIe expansion slot are four white 7-pin SATA connectors. The C612 chipset can support up to 10 but there is limited space on this motherboard and they are squeezed in flanked by two DDR4 SODIMM slots above and the C612 PCH to the left.

Augmenting the internal storage capabilities is a USB 3.0 header. This is perfect for an embedded VMware ESXi boot device. One can also see a USB header next to the internal Type-A connector. Next to these are three 4-pin PWM controlled fan headers.

Looking at the top of the motherboard we can see standard 24-pin ATX and 8-pin CPU power connectors just on the edge of the motherboard. There is another 4-pin fan header just above the CPU power and next to the two DDR4 SODIMM slots.

Here is a shot of perhaps the most inefficient part of the motherboard. The ubiquitous Aspeed AST2400 controller that handles the BMC. The IC just below it and next to another 4-pin PWM fan header is RAM for the Aspeed controller. The primary purpose of the Aspeed AST2400 is to provide OOB management functionality which is certainly a must-have feature. The chip is used across many different vendors so it is the go-to industry option at this point. For a low compute power ARM processor, GPU and RAM, the two IC’s certainly take up a lot of space.

The rear I/O panel of the motherboard has legacy VGA out and two USB 3.0 ports. This is generally enough to work with when one needs physical access. The three RJ-45 ports provide network connectivity. There is one Ethernet port for the OOB IPMI management NIC. The other two NICs are Intel i210 based gigabit Ethernet ports.

Two chips were missing from the top of the motherboard: a Realtek NIC and a Nuvoton controller. Normally, one would not notice they were missing from the top of the motherboard but since we work with so many motherboards, we noticed this immediately. ASRock Rack’s designers did include the IC’s, they were just on the opposite side of the motherboards along with more components than we are accustomed to seeing. The ICs are relatively short so we did not notice any issue with clearances in our various test chassis.

This motherboard is absolutely packed with components and certainly does deliver on bringing high-end compute to the smallest possible package.

ASRock Rack EPC612D4I Build Tips

The ASRock Rack EPC612D4I pushes the envelope in a number of different aspects. First, it uses ECC DDR4 SODIMMs which are relatively new on the market. Innodisk sent two modules at a time when one could not purchase these modules in the US. We previously previewed these modules on STH here.

We do expect more vendors to start offering these modules as Skylake based processors hit the market. They are important because without using the much smaller slots, the mITX form factor is just too small for the big Xeons. We would love to try the system with another 16GB, and hopefully we will get that chance soon.

Being so small, we normally run our thermal testing on the boards. We ran into a snag with this board. Our normal 4U heatsink and thermal testing setup basically obscured the motherboard. Suffice to say, with the 4U cooler our 120w processor was easy to cool. With a 1U passive heatsink like the Dynatron T318A (vapor chamber) cooler we used in this mock up one did need to ensure proper airflow.

Using a mITX motherboard in a standard 1U chassis (one that comfortably fits ATX motherboards) did pose a challenge. For example, the redundant fans in the above mock-up required fan extension cables to reach the shorter sized motherboard. This is not a major issue, but it does add to the list of things to check when using such a small platform. There are plenty of designs out there that work well with a small motherboard. (e.d. and yes, the day glow Velcro tabs are there just for fun!) Our suggestion is that the ASRock Rack EPC612D4I is best suited to mITX enclosures (1U or tower/ desktop).

ASRock Rack Management

ASRock Rack has an industry standard web management interface based on the American Megatrends MegaRAC solution. The ASRock Rack IPMI 2.0 implementation has an array of configuration options and sensor information. One of the more interesting features is the ability to graph sensor information over time.

ASRock Rack includes remote KVM-over-IP functionality. From what we understand, 80% or more of servers are now supporting IPMI 2.0 and some sort of KVM-over-IP solution. The reason is simple. In a datacenter, KVM-over-IP saves companies a lot of time by allowing remote administration instead of having to physically access the server. Companies like HP, Dell, Lenovo and Intel are charging more for full KVM-over-IP functionality, but ASRock includes it with the motherboard. That is by far the preferred way to bundle the functionality.

The ASRock Rack Remote management solution provides the remote administrator with the information they need to manage servers far away from the datacenter.

Conclusion

If you are looking for a mITX Xeon E5 server platform with lots of compute, the ASRock Rack EPC612D4I is simply the best option out there today. Using the narrow LGA2011 ILM, and DDR4 SODIMMs the motherboard is even able to include quad channel DDR4 memory in a very compact package. The form factor is liberating as it can handle a variety of tight chassis, perfect for short-depth applications. The caveat here is that if one does have room for a larger chassis (e.g. at least ATX size) that larger chassis can offer more expansion options in terms of storage, network and RAM. Overall the ASRock Rack EPC612D4I uses its innovative design to bring the Intel Xeon E5 family to extremely compact footprints.