The Supermicro X8DTH-6F motherboard is a substantial dual processor (DP) motherboard loaded with on board features and expansion possibilities. Users looking for a low-end server, or a simple 20 drive system accessed over a single Gigabit Ethernet port should avert their eyes because this board is about as feature packed as they come. For those looking to build a very expandable powerhouse of a server, this is probably a good starting point. I get asked about this board and the X8DT6-F (similar board with a single IOH and fewer PCIe slots) at least once a week so when Supermicro approached me about reviewing the board I decided, albeit daunting, I would check the board out.
Test Configuration
This board presented a bit of a challenge as I have been accustomed to reviewing single CPU (UP) systems over the past year. I put together a test system for a few reviews I am working on at the moment and posted the build log for this dual processor test bed earlier.
- CPU: Dual Intel Xeon E5606
- Motherboard: Supermicro X8DTH-6F
- Memory: 24GB ECC DDR3 1066MHz DRAM (6x 4GB UDIMMs)
- OS Drive: OCZ Agility 2 120GB
- Additional NICs: Intel EXPX9501AT, Mellanox MHEA28-XTC, and Supermicro AOC-STGN-i2S
- Additional RAID Controllers/ HBAs: LSI 9211-8i, IBM ServeRAID M1015, Areca 1680LP
- Enclosure: Supermicro SC842TQ-665B
- Power Supply: Supermicro 665w (included in the SC842TQ-665B)
While another configuration may have been better attuned better to highly CPU intensive tasks, the Xeon E5606 CPUs are fairly power efficient while allowing full use of the boards expansion capabilities.
Board Layout
Dual LGA 1366 CPU sockets are probably one of the most important features of the X8DTH-6F. With this, one can employ everything from dual core CPUs to six core CPUs, with or without Hyper-Threading. Both the 5500 and 5600 series Xeon processors (Nahalem-EP and Wesmere(-EP) respectively) are supported. At this point in time, Intel’s LGA 1366 processor lineup is very mature so options range from the sub-$200 per CPU range to over $1,500 per CPU with multiple speed and core count gradients in between. One note is that upon setup with a Xeon E5600 series CPU, I had to enable AES-NI features in the BIOS as the attribute was set to “Disabled” by default.
The Supermciro X8DTH-6F includes six 240-pin DDR3 slots per socket supporting up to 48GB of unbuffered ECC DIMMs and 192GB of registered ECC DIMMs. The support for significant amounts of memory, per system board, is one key differentiator of the dual-LGA1366 platform over the single CPU LGA 1155 and 1156 platforms. The X8DTH-6F packs the ability to house dual CPUs for heavy workstation or server loads, and those same loads, be they CAD applications to virtualization applications, generally require more memory to run efficiently. Co-incidentally Windows 7 x64 is currently limited to 192GB of RAM (Server 2008 R2 in contrast, is limited to 2TB), so for some Windows applications, 192GB may be a practical limit a this time.
The next defining board attribute are the dual IOH36 controllers and the seven PCIe 2.0 x8 slots (in x16 physical) they support. Practically speaking, this gives an absolutely monster amount of expansion capabilities for each system since there is ample PCIe bandwidth across the board. For applications where multiple RAID controllers, iSCSI HBAs, 10Gb network controllers and GPUs will be used (for Hyper-V RemoteFX or GPU computing), the seven expansion card setup allows for a lot of flexibility.
One often-overlooked feature is the number of 4-pin PWM fan connectors on motherboards. The X8DTH-6F has a full compliment at eight 4-pin power connectors. This is significant because it allows the BMC functions of the Supermicro motherboard to adjust fan speeds based on temperature changes and load changes. When combined with appropriate 4-pin fans, and low power components, this can greatly reduce the average fan speed of the in-chassis cooling and likewise the overall noise profile of the machine. For many small and medium business users, as well as home environments, noise is a key concern.
An important consideration with the X8DTH-6F is that the board is very large, being an EATX design measuring 12” x 13” in size. For smaller enclosures, or enclosures with multiple fans mounted in a center partition, it is important to ensure that the board will fit in a given chassis. I was able to fit the board in the relatively compact Supermicro SC842TQ-665B chassis very carefully, but there was not an abundance of room. The EATX design is a positive one as a whole because it provides for not just the ample expandability and list of integrated peripherals, but also because it makes the amount of room to work on the system manageable, and keeps all items aligned for good airflow, including the 24-pin ATX and dual 8-pin CPU power connectors.
Finally, regarding the expansion slots, the PCIe slot closest to the CPU socket is very close to that CPU socket. I used relatively large 4U heatsink and fan assemblies and they came close to, but did not interfere with the cards installed in the first PCIe slot. On the other hand, the same slot is subject to length restrictions because one of the DIMM sockets is in-line with the same PCIe slot.
Features
The Supermicro X8DTH-6F has a lot of built in features aside from the expandability that the boards layout affords. It has an Intel 82576 dual port gigabit controller, LSI SAS2008 SAS 2 RAID controller and IPMI 2.0 features including KVM-over-IP.
One intriguing feature of the X8STH-6F is the onboard LSI SAS2008 controller. This controller has the ability to connect eight SAS 2 and SATA III drives running at 6.0gbps and is backwards compatible with older generation drives. As was covered in the previous reviews of the LSI SAS2008 based LSI 9211-8i as well as the Supermicro X8SI6-F, the SAS2008 controller is fairly well supported in major operating systems aside from FreeBSD where the support is not yet in the STABLE-RELEASE. One notable FreeBSD derived ZFS platform, ZFSguru maintained by forum member sub.mesa, now supports the SAS2008 based controllers out-of-the-box, but the current 0.1.7 release is still not as full featured as some other options out there.
Although the controller initially ships with IR RAID firmware supporting RAID 0, 1, and 10 there are two additional options. First, a user can upgrade the controller to unlock RAID 5 capabilities using a RAID key. This option is good for users that want low-cost RAID 5, but for performance striped parity setups, I still recommend a proper RAID controller with a battery or flash-capacitor backed write cache. The other option, and perhaps the one most used, is flashing the controller to IT mode. The primary benefit here is that it allows the controller to present disks directly to the operating system, thereby simplifying the control structure of the disks. It is recommended that users of mdadm and ZFS based systems flash the controller to IT mode to increase reliability. One can note that the LSI SAS2008 controller is in addition to a standard compliment of six SATA II ports (with SPGIO headers) and one internal USB header.
An Intel 82576 dual port Gigabit Ethernet Controller is built in. The 82576 is the same controller used in the Intel ET Dual Port Server Adapter, EF Dual Port Server Adapter, and both the ET and ET2 quad port adapters when used in pairs. This is an update to the 82654L used in many previously reviewed boards such as the ASUS P7F-E, Supermicro X8SI6-F, X8SIL-F, X8ST3-F, and the Intel S3420GPLC. Aside from these two Intel ports, Supermicro also includes a Realtek based NIC for dedicated IPMI 2.0 access.
IPMI Remote Management Features
Supermicro’s IPMI and KVM-over-IP as described a few times on this site, allows for a lot of deployment flexibility. Things such as fan speeds, chassis intrusion sensors, thermal sensors, and etc. can be monitored remotely and alerts setup to notify the administrator of issues. Beyond this, the Winbond WPCM450 BMC chip also allows for the remote power up, power down, and reset of the server in the event that it becomes unresponsive. I will note that this setup has been running 10Gb networking tests and Folding@Home for the past six days without needing to use the below buttons for anything other than turning on the system. In fact, the test system has never had a keyboard, mouse, CD/DVD ROM, or monitor hooked up to it, even after multiple BIOS tweaks and three operating system installations.
Another important feature is the ability to remotely mount CD images and floppy images to the machine over the Realtek Ethernet controller. This keeps maintenance traffic off of the primary Intel NICs and at the same time removes the need for an optical disk to be connected to the X8DTH-6F.
In combination with the baseboard level management features just described, KVM-over-IP features of the X8DTH-6F are not to be underestimated. One can log on to the server using either a web browser with Java platform support, or through Supermicro’s IPMIview software and have remote console capabilities, with mouse support included. Oftentimes, users opine that shell access is enough to troubleshoot, but this BMC level access to the remote system allows one to immediately see things such as a hung and incomplete boot process, frozen servers, as well as manage motherboard and add-in card BIOS remotely. I believe that the IPMI 2.0 features, including the KVM-over-IP features are essential for a server-class motherboard unless one has an external KVM-over-IP unit and a network addressable power source. For users with redundant PSUs, as will be the most common use case with this motherboard, remotely toggling on/ off multiple power supply ports across multiple PDUs can be difficult making the motherboard option a bit more elegant.
Conclusion
The dual LGA1366 with dual IOH36 chipsets platform provides for a lot of innate expandability. With up to seven PCIe 2.0 add-in cards, 192GB of RAM, and fourteen on board disk drive ports, on board dual Gigabit ports, and support for up to twelve physical cores, the Supermicro X8DTH-6F provides a solid base that will be able to expand to support many roles. The other side of the equation is price and at around $570 retail, the board is about $200 more than the X8SI6-F. For that $200, the X8DTH-6F offers a similar feature set with the on board LSI2008, dual Intel GigE ports, IPMI 2.0, and etc., but the X8DTH-6F delivers much more in terms of expandability.
Great review! I’m just about to purchase this for my new server. Need all the PCIe slots for various DVB-S2 and SATA cards.
I’ve heard that Supermicro boards are picky about RAM. What is your experience? Do you always use modules Supermicro list as “compatible”?
My god, beast. I wish I could ever justify a board like this. Great review!
Been looking to buy one too. Finally a legitimate review.
Patrick — are you reviewing DP systems now? if so great news. how about a x8dt6-f review and comparison?
>>The Supermciro X8DTH-6F includes six 240-pin DDR3 sockets supporting up to 48GB of unbuffered ECC DIMMs and 192GB of registered ECC DIMMs.
That should read six 240-pin DDR3 sockets per CPU socket. Or alternatively 12 240-pin DDR3 sockets with no mention of the dual sockets?
Great review too, by the way.
All, thanks for the kind words. I will be looking at more DP related items in the somewhat near future as I have gotten a lot of requests.
Thanks for the catch Ryan. I have made the update.
Patrick,
Great review – as always.
Do you know if any of the PCIe slots on this board sit behind a bridge like they used on the X8SIA-F? Or are they all served directly by the 50500 IOPs? Might be important to know if you are planning to use VMWare / ESXi since you can’t do device passthrough from a PCIe slot that sits behind a switch or a bridge.
There’s a block diagram in the motherboard manual (you can download it at the supermicro site). It looks like all the PICe slots are directly connected to the IOHs.
There’s a block diagram in the motherboard manual, and it appears that all PCIe slots are connected directly to the two IOHs.
Can anyone reccomend heatsinks for this board and norco 4224 combination?
I used SNK-P0040AP4 as they have the proper screw style mounts (consumer/ X58 board mounting is different).
There are quite a few options out there, those were just relatively inexpensive and 19.5db.
A really GREAT set of SuperMicro Motherboard reviews (not just this one).
If only you could review a Quad Opteron Board like the new H8QGL-8F+/H8QGL-iF+ (Workstation) or the older H8QG6+-F/H8QGi+-F (Server) G34’s. These will take Bulldozers soon (assuming a BIOS update).
Thanks for the compliment Joe. I will probably not get to quad G34 setups before Bulldozer, but I do have two Opteron 6164 HE’s sitting on my lab desk right now. More to come shortly.
Not a bad review. I’ve had my X8DTH-6F with a pair of E5520’s on it for about 2 years now, (Bought a very early sample board, haven’t regretted it).
Another thing I would like to add though, is that with dual IOH’s, and 7 slots, it can be fairly easy to add one too many cards with an option ROM, and monkey the BMC up to where you have to pull power, and give it a cold boot.
This is why being able to enable or disable option rom support on a PER SLOT LEVEL (as well as for the onboard 82576 and for the LSI 2008) is a godsend.
Now if only there were OSX drivers for the 82576 and the LSI 2008…
I just picked up this board — for me, the IOH runs *really* hot.
I had to add another fan inside the case to blow air across the IOH, otherwise the system cranked up the PWM fans to full speed (which, didn’t help as those were only on the CPUs and at 9500 rpms, sounded like I was standing next to a jet engine).
Once I put the fan across the IOH, all was solved (which, SM’s excellent support helped me to find out).
Of course, this could be a poor case design (Lian Li PC-A70F). I’ve ordered improved fans to install that should significantly increase the airflow, and hopefully I can remove the random mid-case fan. My original pick likely would not have had this issue (a Fractal Design Define XL w/ a side fan) (the only 2 reasonably priced cases to natively hold 10 disks). I plan to install a 5-in-3 expander/backplane to get to the 14 disks that the board can handle.
Also, and I have no idea if this is the case with IPMI or just the SuperMicro limit — you can’t install and enable a PICe graphics card *and* get the remote console to work (since, you can’t run both the integrated graphics and PCIe graphics). This is what SM support tells me at least (and was my experience after installing a card and setting the BIOS to “off board”).
Even with a GeForce 520 GPU I can’t seem to get the Aero theme on my Server 2k8 R2 install working — anyone have any ideas on this? I did install Desktop Experience, but the theme stays grayed out… (I didn’t have this same issue with an Asus M4A78-EM and its integrated Radeon HD3200 chip on another build).
And — whats’s the technique for using the IPMI Remote Control / Virtual Media to attach network ISOs? I got this to work in the Remote Control / Remote Console w/out any issues, but not from the Remote Control / Virtual Media interface (which, I would have preferred, but as long as one of them works, I guess its no issue).
Hey Mike: I looked and did not have a screenshot on hand. All of the SM machines are packed for the move this weekend. Maybe try the forums as it is a pretty easy thing to do from the web GUI.
you should do a series for compute friendly motherboards. ones that allow more than 4 GPUs to be installed and be stable.
I am looking to buy it but wanted to find out the power usage on this motherboard with those CPUs you got? I was planning to run dual L5630 CPUs. Need power efficient home server with a lot of ram.
Has anyone gotten GPU pass-through to work with this motherboard? I have tried using LINUX, PROXMOX and ESXI 6.7 and I can get GPU audio to pass, but never GPU video.
If you have a way please tell me how. Many thanks!