The Supermicro X10SRL-F review is our first UP motherboard review of the Haswell-EP/ Intel Xeon E5-2600 V3 generation. With the increased core counts, 10 SATA 3 ports onboard and a transition to DDR4, there is a fundamental switch in the capability of new UP servers and usability of the arrangement as building blocks. Furthermore, with chips like the Intel Xeon E5-2650L V3 one can fit 1U servers with 12 cores and 128GB+ RAM in a 1A @ 120V power limit given the right PSUs and non-AVX workloads making this segment even more interesting.
Test Configuration
We used a fairly typical UP setup for this type of server and what we have used previously in our Intel Xeon E5-2600 V3 coverage. It should be noted that the Intel Xeon E5-1600 V3 line-up is compelling for those looking for fewer cores and higher clock speeds.
- CPU: Intel Xeon E5-2690 V3
- Motherboard: Supermicro X10SRL-F
- Memory: 8x SK. Hynix 16GB DDR4 2133MHz ECC RDIMMs
- SSD: 2x Samsung 840 Pro 256GB 4x Crucial M500 240GB, 4x Samsung SM843 480GB
- Operating System: Ubuntu 14.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5
Overall, the 128GB configuration is not as dense as one can get with larger 32GB DDR4 RDIMMs and larger LRDIMMs, however it is still a reasonably priced configuration.O verall this is a simple setup but should give us a good idea regarding potential performance. We did use our standard add-in card suite including the Mellanox ConnectX-2 and ConnectX-3 cards, Intel 10 gigabit LAN cards, LSI SAS/ RAID controllers and etc.
Supermicro X10SRL-F Overview
The Supermicro X10SRL-F is an ATX sized motherboard, making it capable of fitting into even many short-depth enclosures. The LGA2011-3 socket sits between two sets of four DDR4 RDIMM sockets for a total of 8 DIMM sockets. One can also clearly see that the socket is a narrow ILM type. See here for a guide to square versus narrow ILM. The airflow is set to be standard front to back configuration.
Probably the biggest feature of the X10SRL-F is its PCIe configuration. Grantley has 40x PCIe 3.0 lanes standard per processor. One can see a total of seven PCIe slots, five x8 physical and two x16 physical. The two x16 physical slots are PCIe 3.0 x8 electrical. There are two PCIe 3.0 x8 physical and electrical slots. Two additional slots are x8 phsyical and can run either in PCIe 3.0 x4 + x4 or x8 + x0 electrical modes. That does give some flexibility to the motherboard and this capability is auto switched based on which slots are populated. Finally there is a PCIe 2.0 x4 electrical in an x8 physical slot. The net effect here is a UP motherboard with 40x PCIe 3.0 lanes exposed and 4x PCIe 2.0 lanes. That is the maximum offered by the platform.
From the block diagram, we can see where the additional PCIe 2.0 lanes from the Wellsburg PCH go, and that is mainly to the Intel i210AT NICs and the AST2400 BMC. The ASpeed AST2400 BMC seems to be more or less standard on this generation of motherboards.
The 6 fan headers onboard are 4-pin PWM fan headers which save quite a bit of power by modulating fan speed to match cooling needs. With the new Haswell-E and Haswell-EP lines, idle power consumption is so low that using PWM fan functions can save significant amounts of power. Along those lines, power is delivered to the motherboard via a standard 24-pin ATX connector and an 8-pin CPU auxiliary power connector.
In line with the modernization of the platform, the X10SRL-F includes internal USB 3.0 headers. Both a Type-A style connector for internal USB keys but also a USB 3.0 front panel connector. The trend toward modernizing the rest of the platform is significant with Grantley compared to Ivy Bridge-EP.
Supermicro includes a total of ten SATA 3 ports. These ports have maximum throughput of 6.0gbps. On the previous generation for Intel Xeon E5 V1 and V2 platforms, there were only two SATA 3 ports. What this practically means is that many 1U and 2U enclosures will not require an additional 8-port HBA to attach a full compliment of SATA drives at 6.0gbps speeds. These extra ports can be a significant source of power savings since HBA’s often can consume ~10w each.
The two yellow ports have corresponding SATADOM power connectors. SATADOMs are very popular in many server and network appliances so one can use the X10SRL-F to quickly attach these modules and corresponding power.
The rear I/O is also significantly upgraded. There are standard VGA and legacy serial ports. The two Springville based i210AT NIC sit next to one another. The third NIC is a Realtek based NIC where the RJ-45 port sits atop two USB 2.0 connectors and is dedicated to IPMI 2.0 Out-of-Band (OOB) management.
Finally there are two USB 3.0 ports on the rear of the chassis. Whereas with the V1 and V2 generation, the primary focus was on USB 2.0 ports, with the Grantley platform we are seeing a majority of ports use the significantly faster USB 3.0 specification. It should be noted that the rear I/O follows a similar layout to the X9 generation of motherboards, so re-use of chassis and I/O covers should be possible in many cases.
Overall the X10SRL-F is a great motherboard for those looking either for a simple compute node or an expandable UP platform. The motherboard is designed using the latest generation components and exposes all of the PCIe lanes possible to the user via expansion slots.
Thermal Results
To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turn on FLIR’s MSX enhancements so we can see components outlined clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images.
As one can see, there are some warmer parts to the X10SRL-F but that is relative. There are no real “hot spots” that might cause concern and all components appear to be well below their operational thresholds.
Software and Management
Supermicro’s IPMI and KVM-over-IP as described a few times on this site, allows for a lot of deployment flexibility. Things such as fan speeds, chassis intrusion sensors, thermal sensors, and etc. can be monitored remotely. Alerts can be setup to notify the admins of issues. Beyond this, the functionality also allows for remote power control. You can do remote power up, power down, and reset of the server in the event that it becomes unresponsive. In fact, the test system has never had a keyboard, mouse, CD/DVD ROM, or monitor hooked up to it, even after multiple BIOS tweaks and operating system/ hypervisor installations.
Another important feature is the ability to remotely mount CD images and floppy images to the machine over the dedicated management Ethernet controller. This keeps maintenance traffic off of the primary Intel NICs. At the same time it removes the need for an optical disk to be connected to the Supermicro motherboard.
The latest revision of Supermicro’s management features include some key changes and new functionality. This includes the ability to recover from a bad BIOS flash from the IPMI interface.
We will have more on this feature soon, but suffice to say this makes management of remotely located servers significantly easier with some safeguard in the event a BIOS flash goes poorly.
Conclusion
Overall the X10SRL-F is a very interesting base platform. It offers equivalent functionality to many previous generation UP and DP servers in a UP platform. It also has its PCIe 3.0 and 2.0 lanes exposed to make it a solid building block for highly customized UP servers. Fitting in a standard ATX size motherboard design means it is also flexible and can fit into most standard enclosures. Overall, it is a solid building block for those who do not need many integrated components.
Do you also have some information in regards to power consumption?
Already wanted to by the predecessor “X9SRL-F”, but now since tehy have released a new board (and since I have not yet bought any of the other components like CPU or RAM), I’m strongly thinking of going for the new platform! ;-)
What makes me curious though, is: Is this board supported “out of the box” for ESXi, or do you have do install additional drivers to get everything (NICs?) to work?
Also, does direct Path I/O work flawlessly with this configuration?
Also, ESXi 5.1 is not on the OS support list, I wonder if it would still work with ESXi 5.1…
i second this, a kill-a-watt load at idle would be most delightful
I have a similar board the x10slf7 which uses the same nics, i210at using the esxi 5.5 update 2 image, they are included automatically
I’m working on a build right now with a MBD-X10SRI-F and I’m running into all kinds of issues, mainly with getting it to boot. You seem to have had more success than me getting yours to work and if you’ve got any tips that would be great!
I’m using a E5-2620v3 and 16GB of Crucial DDR4.
Thanks
I’m pretty sure I had a bad power supply which now may have fried the motherboard… Thanks Newegg!
Enjoyed your excellent review.
I am planning to buy Supermicro X10SRL-F.
This has narrow ILM.
Can you suggest a couple of best quality/performance heat sink models for 2U chassis.
My CPU will be E5-1650 v3
I agree, an idle power consumption test would be awesome. Apparently no one tests power consumption of the e5 v3 series CPUs or motherboards. I keep reading that they only use a few watts more at idle than an Avoton, but there are never any hard numbers proving it. I’ve googled a lot, and found nothing.
Really – Googling Xeon E5 V3 power consumption had an article I wrote come up as one of the top 3 results: http://www.tomshardware.com/reviews/intel-xeon-e5-2600-v3-haswell-ep,3932-9.html which deals with power consumption.
Thank you Patrick. I’m not sure how I missed that, but you are a gentleman and a scholar.
Is it possible to fill all the slots of this card with SAS-cards? Say 2×9202-16e and 5x m1015?
is it possible to install 40 graphic cards on this board for mining rig?
the gpu just need 1x lanes in mining
see this topic
http://www.pcgamer.com/how-many-gpus-can-you-fit-in-a-single-system/
that is a great article over at toms HW, however i would like to see power consumption on this config and (more so) with a lower end cpu. (fwiw most of patrick’s more recent articles do have this type of data). im currently trying to researching balancing cost$ and power usage between these types of x10 boards and the x11 xeonD boards. tks
James, I would love to do it but it is harder to go back and do older hardware like this. The good news is that as STH grows we create a bigger dataset for the future
Truly rather great web site. Rather good towards browse and very a great deal interesting content. Certainly period incorporates not nevertheless appear again and greet.
“There’s definately a great deal to know about this issue.
I like all the points you have made.”
If some one wishes expert view concerning running a blog afterward i suggest him/her to visit this weblog, Keep up the pleasant job.