Supermicro X8SI6-F Motherboard Review Including Onboard LSI SAS 2008 Controller

Oftentimes those building their own small business and home storage servers are looking to save considerable amounts of money. In the process, users oftentimes look to the lowest cost motherboard they can find, with the wisdom that add-in cards can get the features that they want. On the other end of the spectrum are server motherboards like the Supermicro X8SI6-F that have everything or almost everything users need onboard. The Supermicro X8SI6-F costs in the $300 range, which would be a turn-off for many users, but when one looks at the features, $300 looks like a relative bargain.

Test Configuration (Base):

1. CPU(s): Intel Core i3-530 and Intel Xeon X3440
2. Motherboard: Supermicro X8SI6-F
3. Memory: 2x 4GB kits of Kingston ECC 1333MHz DDR3 KVR1333D3E9SK2/4G (Unbuffered)
4. Case: Norco RPC-4220
5. Drives:  8x Hitachi 7,200rpm 2TB, 2x Western Digital Green 2TB, 6x Seagate 1.5TB 7,200rpm
6. OS(es): Windows Server 2008 R2 with Hyper-V installed, Windows Home Server (in Hyper-V), Windows Home Server V2 Codename Vail (in Hyper-V), NexentaCore 3.0.1, OpenFiler (v2.3), FreeNAS (0.7.1), VMWare ESXi 4.1

Features Overview

The LGA1156 platform has been a strong platform for enthusiasts and is quickly catching on with HTPC users for its solid performance and low power consumption. Motherboard manufacturers like Asus, Gigabyte, MSI, and others all have enthusiast level boards with the LGA1156 chipset with multi-stage power (for overclocking), NF200 switched PCIe lanes, USB 3, SATA 3, onboard HD audio, HDMI outputs and etc. Some LGA1156 boards also include dual Realtek gigabit NICs although a single NIC is standard. Frankly, most if not all of these features are of low value in storage servers.

In a storage server, one typically looks for a few features:

1.      Enough CPU power to perform any parity calculations required
2.      Connectivity for multiple hard drives
3.      Management features
4.      Intel network ports
5.      Compatible devices for a range of server operating systems
6.      Enough RAM for cache purposes (especially important for ZFS caching)
7.      Solid customer support and reliability

With the first item above being almost a given assuming a user properly sizes their CPU (few storage servers built on this platform will need a six core CPU), the remaining items will be the focus of this review.

Hard Drive Connectivity and the X8SI6-F… Awesome!

If there is one area that the X8SI6-F excels in, it is clearly in connecting hard drives. The X8SI6-F has an onboard Intel ICH controller as part of the 3420 chipset that provides six SATA 2.0 ports. This is a fairly standard feature on server motherboards. One will not find an eSATA port on the backplane because typically rack mount enclosures are large and internal storage is much preferred. Although the Intel ICH is a great controller, and does does support RAID functions, operating systems can have issues with the ICH in RAID mode. While Windows operating systems have no issues, operating systems such as OpenSolaris/ NexentaCore do not work well with “fake RAID” implementations found on chipset RAID solutions. Setting the controller to a non-RAID mode brings compatibility in line, and with the excellent latency characteristics, makes both hard drive performance and SSD performance (very important for cache/ L2ARC drive performance).

Going beyond the simple ICH, the X8SI6-F provides eight SAS 2.0 (6.0gpbs) ports onboard courtesy of a LSI SAS 2008 chip onboard. Connectivity is provided through two SFF-8087 ports which make cabling in chassis like the popular Norco RPC-4220 and RPC-4224 very clean. Windows Server 2008 R2, Windows Home Server, various Linux flavors, OpenSolaris/ OpenIndiana/ NexentaCore and VMware ESX and ESXi all work really well with the LSI SAS 2008 controller.

For all intents and purposes, the Supermicro X8SI6-F’s onboard LSI SAS 2008 controller is equivalent to the LSI 9211-8i add-in card. Just for reference, the LSI 9211-8i retails for around $260 US. The important thing here is that even if the onboard controller did fail, one could purchase a compatible add-in card and recover the array. Just to test this, using Windows Server 2008 R2 I was able to unplug drives (all idling) from the onboard controller and into a LSI 9211-8i controller without the OS and was able to access the drives seconds later.

Aside from the obvious cost savings, both implementations support the very popular HP SAS Expander. Connecting one of the Supermicro X8SI6-F’s ports to the HP SAS Expander yields 32 SAS/SATA ports (through the SAS Expander), 4 additional SAS/ SATA ports (through the second SFF-8087 connector which can also be used with a second expander), and then 6 SATA ports via the Intel ICH. Adding one SFF-8087 to SFF-8087 cable and a HP SAS Expander gives connectivity for 42 drives! One could also either daisy chain the SAS Expanders or connect a SAS Expander to the second of the Supermicro X8SI6-F’s SAS ports and have connectivity for 70 drives.

One typical question I see on forums is how does the SAS 2008 chip perform with mechanical drives and a HP SAS Expander. Here is a preview ATTO Benchmark showing over 900MB/s to 1GB/s in read speeds which is more than adequate for many file servers. As a spoiler for an upcoming feature, the performance numbers are virtually identical to the discrete LSI 9211-8i’s performance.

For users looking to build systems with less than eight drives, this is not an issue. For users looking for a solution that can scale inexpensively, the X8SI6-F provides a clear value proposition.

Management Features

One of the things that I consistently offer advice on is management features on any platform intended for server use. Ideally, a storage server’s only maintenance should be with its hard drives. Mechanical drives fail, and sometimes capacity needs to be expanded. Unfortunately, sometimes things go awry. A CPU heat sink clip may dislodge causing a CPU overheat situation. A kernel may panic causing the inability to login to a machine. An error prior to OS boot may cause a machine to become totally unresponsive. For all of these situations, server grade motherboards have features to help an administrator recover from a potentially negative situation.

I have discussed Supermicro’s IPMI implementation with KVM-over-IP at great length lately, including the highly recommended utility IPMIview. Like the other Supermicro “-F” boards reviewed here, the X8SI6-F’s IMPI 2.0 implementation utilizing the Nuvoton WPCM450 Baseboard Management Controller (BMC)  includes ability to mount CD’s and DVD’s over the onboard dedicated 10/100 Realtek controller. The KVM-over-IP pulls a bit more bandwidth (<2mbps from what I have seen) than RDC, so being able to offload that traffic is not essential from a performance point of view, but it is welcome. Furthermore, the additional NIC allows the management interface to be placed on a separate management network which is a best practice considering that the BMC allows for remote rebooting and local console access over a network interface.

Bottom line, the IPMI 2.0 and KVM-over-IP functions on the Supermicro X8SI6-F and similar motherboards are simply must-haves for anyone except those running a server next to their desktop, and even then it is a nice to have. Recently I had a Hyper-V VM bring down a Windows Server 2008 R2 server and I was able to reboot the machine from my car en-route to dinner. It was the difference between having to miss dinner due to an unscheduled stop to the server (which was a 35 minute drive away each way) and a 90 second stop in the parking lot of the restaurant. Those situations are where building servers off of consumer grade hardware clearly do not pay off.

Intel Network Ports

Supermicro’s X8SI6-F, like the X8SIL-F it has two Intel 82574L based gigabit NICs onboard. Anyone reading my site will note that I am biased heavily in favor of Intel based gigabit NICs as they are the most consistently high-performing NICs that I have used. Furthermore, the compatibility of the 82577L with Windows, Linux, FreeBSD, VMware ESX and ESXi, and OpenSolaris is a given. The OS and Hypervisor support is generally very good for these Intel gigabit NICs. Second, I have seen much better throughput on Intel NIC to Intel NIC gigabit networks versus networks with Realtek and Marvell network controllers. After months of using Realtek NICs in my servers, the cost savings were just not worth it.

The Intel 82574L supports features like TCP/UDP CPU offloading and TSO for lower CPU utilization, PXE/ iSCSI boot, Jumbo frames, two transmit and receive queues per port, a 40KB packet buffer, and etc. For reference, the Intel 82574L is the same controller used in Intel’s Gigabit CT (EXPI9301CTBLK) PCIe x1 network adapter. In essence, the Supermicro X8SI6-F and similar motherboards save one $60-70 and a PCIe slot (if one were to get a dual port Intel NIC) or two versus a lower-cost board where NICs would be required separately. Another important note is that the onboard NICs do not require a raised PCB expansion card for network connectivity which helps airflow and also in 1U and 2U enclosures where expansion ports are limited, oftentimes requiring riser cards.

Device Compatibility

After installing, seven different operating systems/ hypervisors over the past ten days, the device compatibility is great. By that I mean that every operating system did not require searching for drivers in order to get all onboard components to work. The only issue was with FreeBSD/ FreeNAS installations which do not have default LSI SAS 2.0 drivers except for FreeBSD 9-Current.

With how finicky both OpenSolaris variants and VMware ESXi can be with drivers, having something that simply works out of the box is great. The Asus P6T7 WS Supercomputer, for example, was the exact opposite where ESXi 4.0 gave multiple compatibility errors during installation necessitating the use of add-in cards.

Memory Compatibility

The Supermicro X8SI6-F is based on the Intel 3420 chipset. Paired with an Intel Xeon X34xx series CPU, the X8SI6-F supports up to 16GB of unbuffered ECC DIMMs in four slots. With registered ECC DIMMs, the X8SI6-F supports up to 32GB of memory. For users that are willing to risk their data to non-ECC DIMMs, the 16GB limit also applies. It should be noted that like other motherboards based on the 3400 series chipset, it is strongly recommended to use memory that are on the hardware compatibility list.

Internal USB Headers

A really nice feature of most server motherboards is the presence of an internal USB header. The Supermicro X8SI6-F has two such headers, much like the Supermicro X8ST3-F. This allows one to add two USB thumb drives to the interior of the enclosure housing the Supermicro X8SI6-F. For home and small business storage servers, this is an enormous benefit. FreeNAS, Openfiler, unRaid, untangle and many other operating systems can install directly to a relatively low-cost flash drive saving an onboard SATA port. Installed in the server’s enclosure, the USB stick will the USB drives are unlikely to become dislodged for any reason. With two such ports I have made it practice on my X8ST3-F based servers to have a secondary 2GB drive with a recovery Linux distribution in the event something goes severely awry. Having another boot option for the $20 or less cost of a USB drive has come in handy, especially when combined with the KVM-over-IP features of the X8SI6-F.

Expansion Slots

Expansion slot wise, the X8SI6-F has a two PCIe slots, one a PCIe 2.0 x8 slot and another PCIe x4 electrical in an x8 physical. For legacy cards, there is a PCI slot onboard also. One will note that this is one slot less than the X8SIL-F, which is due to the LSI SAS 2008 controller occupying those PCIe lanes. Unlike a LGA1366 platform, any LGA 1156 platform is going to have less available PCIe lanes (unless a PCIe switch is implemented and those are usually not recommended in server environments.)

Suffice to say, one is sacrificing a PCIe slot for an onboard SAS 2.0 RAID controller, but for anyone looking to purchase the X8SI6-F, this is a known and welcome tradeoff.

ATX Form Factor

In contrast to the mATX Supermicro X8SIL-F, the X8SI6-F is a full sized ATX motherboard. This has advantages and disadvantages. One advantage of this form factor is that it allows the SAS ports to line up very well with many enclosures that have front facing disk headers in the front left (looking at the drive bays) of the enclosure. Using 1ft cables I was easily able to connect the X8SI6-F’s SAS ports to both the Norco RPC-4220 drive backplanes as well as a HP SAS Expander sitting in the bottom PCIe slot (x4 electrical). The main disadvantage is that this will not fit if someone is looking to make a DIY JBOD DAS-SAS Expander enclosure as I provided a guide for earlier. I would, however, offer that this board is more appropriate for a user looking to make a head unit instead of a DAS disk shelf.

As one can see above the ATX form factor of the X8SI6-F is significantly larger than that of the mATX X8SIL-F.

Bundled Accessories

Supermicro provides a nice package of six SATA cables and four SFF-8087 breakout cables. This means that for users installing the X8SI6-F in conventional tower cases or other chassis with individual backplane ports, the cables are there to support fourteen drives immediately.

The motherboard manual does a good job of explaining a lot of the jumper configurations for things like disabling onboard LAN, video and SAS. One will note that the IO back plane is actually the same as the X8SIL-F which is a big plus if you swap motherboards ever.

Supermicro Quality and Support

Supermicro competes in the enterprise space and therefore has great support for technically inclined users. For example, after receiving a X8SIL-F PCB Revision v1.01 which did not work with the Intel Core i3-530 I had installed, I contacted Supermicro support about this issue. It was a known incompatibility and Supermicro was willing to immediately exchange my board. Supermicro is one of those companies that I have found over the past decade that if you do your due diligence before contacting support (double-check everything and do a quick Google search), they do a great job helping you. Overall, I have installed approximately 50 Supermicro based systems in the past 18 months and have yet to experience a failure. A small sample size for sure, but it is the only measure of reliability that I have to go by.

Value Proposition

When looking at a $300 motherboard, many users may automatically dismiss the hardware as overly expensive. On the other hand, the Supermicro X8SI6-F has an onboard equivelant of a $260 LSI 9211-8i and dual Intel Gigabit CT (EXPI9301CTBLK) network adapters ($35 each). Just those three components alone would cost more than $330 if purchased separately. The caveat, of course, is that one wants to use Intel network adapters (highly likely) and a LSI SAS 2008 based SAS controller. If this is the case, then the price of the motherboard is, in fact, less than the cost of the discrete components. Compared to adding these components to a consumer level motherboard, this solution presents a clear cost savings. With the exception of those users running FreeBSD versions prior to release 9-Current, the value of having all of the KVM-over-IP and IPMI 2.0 features is priceless.

Conclusion

Supermicro offers a compelling platform for anyone using an operating system to manage storage pools and redundancy the X8SI6-F is a tremendous value. While some may consider a $300 motherboard to be “expensive,” the integrated components effectively cost more separate than the entire motherboard. What the X8SI6-F does deliver on it the ability to attach a very large number of drives at a very low cost. Unless one is looking for a hardware RAID 6 solution, the X8SI6-F is perhaps a category killing product for LGA 1156 based storage servers.