Recently Gigabyte sent their MD70-HB0 motherboard over for review. The Gigabyte MD70-HB0 is a dual socket LGA2011-3 motherboard that accepts up to two Intel Xeon E5-2600 V3 processors. We originally spotted the board in our Gigabyte Haswell-EP motherboard release piece. Since then we have had a significant amount of time to spend with the platform. This is certainly a higher-end motherboard option so there is lots to go over in the review. Aside from being based on the newest Haswell-EP platform including the Intel C612 PCH, the platform also includes best-of-breed solutions like an integrated dual port 10Gbase-T NIC and a LSI SAS3008 onboard. The motherboard is already available in many parts of the world and is expected to hit major US retailers shortly.
Test Configuration
We used a fairly typical DP setup for this type of server and what we have used previously in our Intel Xeon E5-2600 V3 coverage.
- CPU: 2x Intel Xeon E5-2690 V3
- Motherboard: Gigabyte MD70-HB0
- Memory: 8x SK. Hynix 16GB DDR4 2133MHz ECC RDIMMs
- SSD: 2x Samsung 840 Pro 256GB 4x Crucial M500 240GB, 4x Samsung SM843 480GB + 6x SanDisk Lightning SLC 200GB SAS and 2x SanDisk Optimus MLC 400GB SAS
- Operating Systems: Ubuntu 14.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5, CentOS 6.5
The test platform was rather expansive. There was one point while running through our compatibility testing where we had a second HBA with 8 drives, an Intel Fortville NIC, Mellanox ConnectX-3 FDR NIC, an Intel DC P3700 NVMe SSD and a Fusion-io 160GB SLC PCIe SSD installed in the system. Talk about a slow boot!
Gigabyte MD70-HB0 Overview
First off, the Gigabyte MD70-HB0 is a large EATX/ SSI EEB motherboard measuring 13″ x 12″ in size. Several years ago motherboards this large were less common, making fitment a potential challenge. These days, such large motherboards are accepted by almost all chassis except the short-depth variants making it fit in the majority of cases.
As one can see, the eight DDR4 DIMM slots per processor are aligned so airflow will go through CPU0 and warm CPU1. With today’s fans this is less of an issue but something to keep in mind when cabling 1U servers around this processor configuration.
In terms of expansion there are three PCIe 3.0 x16 slots and three x8 slots. These can run in either 3x PCIe 3.0 x16 and 1x PCIe 2.0 x4 in a x8 slot. Alternatively using a PCIe switch 1x PCIe 3.0 x16 (x16 bus) slot
2x PCIe 3.0 x16 (x8 electical), 2 x PCIe x8 (x8 electrical) and 1x PCIe 2.0 x8 (x4 electrical) slot.
In addition, there is a copper heatsink that keeps the Intel X540 10Gbase-T controller cool. 10Gbase-T controllers run significantly warmer than SFP+ alternatives so the addition of the higher-end copper heatsink is excellent. 10Gbase-T does offer one major benefit compared to SFP+ based interfaces, it can integrate easily into existing gigabit or 10-gigabit networks using inexpensive and often existing cabling.
Beyond the add-on 10Gbase-T Gigabyte adds a LSI-Avago SAS3008 controller. The sample we received came default in IR (RAID) mode which can be useful for running disks in RAID 0, 1 or 10. Furthermore, the 8-port 12gbps SAS controller can connect to eight SAS 12gbps drives or SAS expanders. LSI-Avago’s newer 12gbps expanders can even use the 8x 12gbps lanes to provide a 96gbps back haul using 6.0gbps drives. That is a major improvement over previous generations which were limited to the lowest common speed in the system.
There are a total of ten 6.0gbps SATA3 ports along the bottom of the motherboard which are from the Intel C612 PCH. Combined with the LSI-Avago SAS controller, the motherboard has 18 total storage ports which is enough for most 1U, 2U and 3U chassis.
Gigabyte also takes advantage of the new C612 PCH’s USB 3.0 capabilities and adds an internal Type-A header for USB keys. A common use is to install VMware ESXi as a hypervisor on a USB key. The attached drives are then used for virtual machine storage. USB 3.0 drives are faster so for the rare reboots, this is a great feature.
In terms of the rear I/O panel there are are two legacy ports, a 9-pin serial port and a 15 pin VGA port. Next to these are the two 10Gbase-T ports providing high-speed networking. On the right side there are two more USB 3.0 ports and the IPMI network jack atop.
Overall the platform has excellent storage features with 18 high-speed storage ports and 10GbE built in.
Thermal Results
To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turn on FLIR’s MSX enhancements so we can see components outlined clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images.
As we would expect, there are a few extra hot spots on the motherboard. Integrated LSI HBAs and Intel X540 10Gbase-T NICs are some of the warmest components on the motherboard. Still, the warmest spot in our test bed was under 44C so well within operational tolerances. Gigabyte’s inclusion of a copper heatsink on the Intel X540 does seem to keep the component cool. One can also see that the second CPU area is warmer than the first, this is expected behavior with higher-power processors and in-line configurations.
Gigabyte – Avocent IPMI Interface
Using this motherboard and testing it through VMware ESXi 5.5, Windows Server 2012 R2, Hyper-V Server 2012 R2, Ubuntu 13.10 and CentOS 6.5 required many operating system installations. We also needed to reboot the system many times during our card compatibility testing where we cycle through various cards such as Mellanox ConnectX-3 VPI 56gbps Infiniband/ 40GbE and LSI SAS 3008 controllers. The Avocent IPMI solution is simply great. After a few weeks of testing it was very stable and more responsive than a standard AMI MegaRAC solution common in the industry.
For those wondering, we did a full two part Gigabyte IPMI walk through (see Part 1 and Part 2). Key features include the ability to remotely mount multiple images over the network and iKVM features.
After spending some time with the Gigabyte-Avocent MergePoint solution it is certainly one of the better web interfaces out there remaining responsive during our testing. Lenovo uses a similar Avocent based system for their server OOB management so it is a relatively mature solution.
Conclusion
Overall this is an excellent motherboard by Gigabyte. Hopefully they continue to build out their offerings. One thing I would like to see is a common rear I/O format that is compatible with existing enclosures. Right now for 2U and larger enclosures, this is not an issue. For 1U enclosures that do not have a standard IO shield opening, this does limit options slightly. On the other hand, most of these motherboards will likely be sold with enclosures and many will be 2U or larger so this is likely a non-issue.
In terms of value, an Intel X540-t2 adapter currently costs more than $500 and a LSI SAS 9300-8i costs about $350. The Gigabyte MD70-HB0 will have a MSRP of under $700 so while it is a higher-end motherboard, the price is probably 35%-40% lower than buying a comparable motherboard, plus two add-in cards. Overall, the high-level of top quality integrated components (Intel X540 and Avago-LSI SAS3008) and management features (Avocent) combine to make an awesome package.
And it’s blue…
Nice looking motherboard tho. I saw the price and was totally freaked out but then realized you are right, those components are worth more. The tradeoff is that you can’t take them to the new machine but:
Motherboard = 350
X540 = 500
SAS = 350
Total = 1150
You’re saving maybe $450 with this. On two motherboards you’d spend $1400 with two of these or $850 for the add-ins and $700 for the motherboards so you’re still ahead.
You’d really only come out behind if you used an add-in X540 and SAS on a third system.
And again not a word about power consumption…
Power consumption is less of a function of the motherboard on these platforms which is why that information is not our motherboard reviews. With Haswell-EP most of the power delivery moved on die so there is even less of a motherboard impact on power consumption in the V3 generation than the V2 generation. Cooling, SAS drives and processors are all a bigger impact on these platforms than motherboards.
For a dual socket system with say 4 SATA drives and 8 DDR4 DIMMs and decent cooling you can expect 80-90w at idle. This platform falls within that range. Under load, the processor choice is the biggest factor.
Patrick,
Agreed with the motherboard being less of a function of power than the cpu – however those of us with outrageous power rates (i.e. SoCal Edison with .32 cents a killowatt in tier 4) a small amount of power delta at idle can translate into large differences in our bill (given that most of our home labs are running 24×7). With the motherboard already on the rack testbed, I think that many of your readers would greatly appreciate you getting us some power consumption numbers – particularly at idle.
Keep up the great job!
Dan
This particular setup, without the SAS SSDs (which use a lot of power) and with only 4 SATA SSDs was pulling about 83-88w at idle. 2x 4U active coolers and 6x 80mm chassis fans.
Nice board dude. I want a XL710 based motherboard next
THIS! Is my kind of motherboard!
Just need a Supermicro 4U case with SAS3 expander now.
Thanks for the update!
I’m sorry for sounding so ungrateful in my first comment. I guess you spoiled us so much with power consumption stats in the past, that we take it for granted :-)
Patrick,
I’m curious, did you test if ESXI would natively install and boot to the internal USB 3.0 a port?
and how much did that affect speed?
my hosts take a couple minutes…. kind of odd since last I checked esxi was only 100mb… makes me think the interface wasn’t the limiting factor
I would like to see one of these, identical in every way expect the sockets, only one socket please.
Dear Kai, check out the MW50-SV0. that is a nice new full-feature single socket board i can recommend. We sell this in europe, that is nice one.
br Jan
Where can this motherboard be purchased? I have look high and low for the this board, but specifically the MD30-RS0? I am feeling frusterated!
Hello,
thank for this great article.
But, it is very difficult to have a answer about this : how many maximum RAM in this motherboard ?
4Go in each slot ? Or 64Go in each slot ?
Regard.
Piet