Today we get our first look at Supermicro’s TwinPro SuperServer 2028TP-DC0FR. This server offers compact high performance features in an 2U, 2x DP, 2-node server with 4x NVMe and 8x SAS3 12gbps drives per node. These systems offer support for NVMe 1.2 and include expanded support for Intel SSD Data Center Family P3700/P3600/P3500 and AIC PCIe 3.0 SSD’s. Beyond storage, the TwinPro series utilizes a shared power, cooling and chassis configuration for increased efficiencies over a standard 1U layout. We do use an older version of a Supermciro TwinPro (for E5 V1/ V2 processors) in our Las Vegas datacenter because the design is efficient and easy to service.
Supermicro TwinPro 2028TP-DC0FR Specifications
- Processor: Up to two 18-core Intel Xeon E5-2600 v3 series (up to 145W TDP)
- Memory: 16x Slots DDR4, Up to 1TB ECC LRDIMM, 512GB ECC RDIMM
- Expansion Slots: 1x PCIe 3.0 x16 slot, 1x PCIe 3.0 x8 slot
- Storage: 12x 2.5″ drive bays, 4x NVMe and 8x SAS3 12gbps drives per node.
- Dimensions: 17.25” x 3.47” x 28.75”
- Weight: Starting at 85 lbs
- From Factor: 2U Rack Mount
Raid Support options:
- LSI 3008 SAS3 (12Gbps) controller (8 Ports); RAID 0, 1, 10 support
Network Controllers:
- Mellanox ConnectX-3 FDR Infiniband 56Gbps Controller
- Intel i350-AM2 Dual Port Gigabit Ethernet, 1x Remote Management Port
Supermicro SuperServers or TwinPro’s come in several different configurations, these include different storage load outs to meet user requirements. In additional the DC1FR TwinPro’s include support for GPU/Xeon Phi’s (w/ GPU kit).
Supermicro TwinPro 2028TP-DC0FR SuperServer
The server we will be looking at today is the Supermicro 2028TP-DC0FR which comes with 24x 2.5″ hot-swap SATA drives up front. The TwinPro itself is divided into two nodes. Each node controls twelve of the twenty four drives.
At each side of the 2028TP-DC0FR has a control panel to power on or off and status LED’s for that node.
All 24 of the drive bays are hot swappable and easy to remove. Push in the red button to unlock the drive and then simply pull the drive out.
Top of the server access is provided by a hinged lid that opens to gain access to the servers cooling fans. Each node has 2x 8cm heavy duty PWM fans with optimal fan speed control to provide cooling. The units simply lift out for replacement if needed. These fans are larger than are found in standard 2U Supermicro servers and can move a great deal of air. They are impressive fans and cool the server very well.
Each of the nodes can be removed out the back of the server. It is a bit of a trick to unlock these, there are two metal tabs just below the pull out grips. It takes two fingers for each side, one pushes in the locking tab and the other pulls on the pull out grips. This method is not all that easy to do and we can see where it might be possible to break off the tabs if flexed around too much. It also takes a fair amount of force to break the seal and pull each node out. This is not all that hard to do when the server is at waist level but could become more difficult when installed in lower or higher rack positions.
One benefit of this setup is that each node can be accessed without having to open the lid of the chassis. That makes CPU, RAM and PCIe expansion cards very easy to service.
In between each node we find the power supplies which are 1280W Redundant High-efficiency Digital Power Supplies w/ PMBus 1.2. These are much easier compared to the nodes to remove if needed.
The motherboard used in each node is the X10DRT-PIBF. This is a full featured motherboard but does have limited PCIe slots given the size of each node. You can install an impressive 1TB of ECC LRDIMM’s in this compact space. Power and storage connections are provided through the riser card on the side.
Each node includes 2x USB 3.0 ports, 1x Remote Management port, 2x Gigabit LAN ports and 1x Mellanox ConnectX-3 FDR Infiniband 56Gbps port and a VGA port.
Here we get a look at each of the nodes on the 2028TP-DC0FR. Large 2U passive heat sinks dominate each node and DDR4 modules fill up the sides.
Even in this dense environment the cooling fans provided enough air flow to keep each node cool, even under heavy loads.
BIOS and Remote Management
Access the BIOS and using Supermicro’s remote management is just like any Supermicro server, it’s simple to do and has all the features you would expect to find.
Using the web GUI one has access to a number of configuration options, telemetry/ sensor information, and remote management tools. Much of this information can be accessed via ipmitool, Supermicro IPMIview (desktop) or the Android/ iOS versions of IPMIview.
For remote management simply enter the IP address for the server remote management port into your browser and login.
The default login information for the Supermicro 2028TP-DC0FR:
- Username: ADMIN
- Password: ADMIN
We ran our tests completely through onboard iKVM which is similar to how one would remotely administer the system in a datacenter.
Test Configuration
The Supermicro 2028TP-DC0FR does not include a DVD drive for installing software so in our case, we used a USB DVD drive to install Windows Server 2012 R2 and run our Ubuntu Disk directly off of the USB drive. We had no issues getting a OS installed on the system, drivers were used directly from Supermicro’s website.
- Processors: 2x Intel Xeon E5-2680 V3 (12 cores each) for each node
- Memory: 16x 16GB Crucial DDR4 (256GB Total) for each node
- Storage: 1x HGST 300GB Hard Drive for OS, 3x additional drives are installed on each node.
- Operating Systems: Ubuntu 14.04 LTS and Windows Server 2012 R2
With 16x 16GB DDR4 memory sticks installed we saw memory speeds of 1866MHz with performance that matched many Supermicro systems we have tested, there was nothing unexpected here and performance was very good.
Memory Latency ranged at ~97.4ns and our average systems using 16x 16GB DIMM’s ranged about ~78ns.
Stream results also compared well to other Supermicro boards, motherboards like the X10DRL-CT series are somewhat faster in this benchmark.
Our CPU2006 test results are also impressive for a system like this. Single threaded results are strong resulting from good stock speeds of 2.5GHz and a Turbo speed of 3.3Ghz. With 24 cores / 48 threads we also get good multi-threaded results with this system. If we had used E5-2699 V3’s we would expect to top out with results for Int-Multi of 520 and FP-Multi of 630, more cores equal higher computational power but with a higher power use and cost also. We think the E5-2680 V3’s are a good match for this system and considerably cheaper cost vs the larger core count CPU’s.
One can find an example of the Linux-Bench performance of this setup here: Supermicro TwinPro 2028TP-DC0FR Linux-Bench Test Results
Power Consumption
We stressed the Supermicro TwinPro 2028TP-DC0FR to get an idea of how much power these units can consume.
The average idle power use of the 2028TP-DC0FR was ~157watts which includes 4x 2.5” d drives and we did peak out at 441watts with ADIA64 Stress Test that we use for load testing. Our results are for each node so they need to be doubled for a complete server. That would bring a total of 882watts for the full 2028TP-DC0FR server. The boot process would see about 250watts peak for each node, the complete system can pull over 500watts on bootup.
We were also interested in seeing what the compute power of this system was capable of so we installed the WCG (World Community Grid) client on both nodes and let it rip for a week or so.
The results shown are for one node only and you can see we peaked out at 170,817 points per day, double that for 341,634 points per day which is insane for a single system. The draw back was the system runs at full 100% load 24/7 using 850watts of power, lots of heat out the back and loud fans.
Conclusion
The X10 TwinPro platforms from Supermicro offer clear performance increases with the latest Haswell E5 CPU’s, increased core count, lower power draw and DDR4 memory. Performance numbers we saw from our tests were of no surprise to us. Moving to a shared chassis form factor motherboard in these systems did not put a damper on any test that we ran compared to other Supermicro X10 platforms we have tested. The build quality of the complete system is high with only a few complaints from us. We would like to see a better design for the node locking system in the back. Pulling the nodes out is not as easy as it should be and there is a risk of breaking the locking tabs and finger grips. After you do this a few times you get used to how they work but it is still easy to use to much force on these parts. It also takes a fairly good tug on the node to pull it out once it is unlocked and this is done with two fingers only. We would also like to see more robust built-in network connectivity with at least dual 10GbE/ 40GbE ports in the back
The Supermicro 2028TP-DC0FR is a breeze to setup and run which we had no issues with and remote management with iKVM included on the motherboards makes the install easy for anyone used to running Supermicro platforms. Unlike systems from other vendors where we have to worry about including a iKVM module or whether our remote management module has the correct firmware to even begin the remote session, Supermicro includes these features standard.
The Supermicro TwinPro 2028TP-DC0FR is a very powerful server and we have only begun to dig into the capabilities this new machine has to offer. We hope to run some additional VM benchmarks and perhaps get an all flash based system running here soon.
