Today we have a review that is a long time coming. A few years ago, iXsystems asked me about an upgrade to their FreeNAS Mini XL line, and I had a simple request: “Denverton.” For those not familiar, Denverton was the codename for the Intel Atom C3000 series. The new chips added more PCIe and storage connectivity, significantly more compute capacity, 10GbE networking, QuickAssist, and higher RAM capacity. For a low-power always-on storage device like a FreeNAS Mini XL+, this is a perfect fit. Last week Cliff covered the FreeNAS Mini E and FreeNAS Mini XL+ launch and today we have our review of the FreeNAS Mini XL+. This is an 8x 3.5″ bay plus 2x 2.5″ bay storage appliance meant for running FreeNAS at the edge.
In our FreeNAS Mini XL+ review, we are going to show you around the unit, and some of the key features. We are also going to show you some of the customizations that make this different from a typical self-built unit. FreeNAS is an open-source NAS distribution, so one can make their own appliance. At the same time, at STH we think it is good practice to support the companies that develop these packages. We wanted to take time showing what the company did, and what we think can be done better in the future.
FreeNAS Mini XL+ Overview
At STH, we are going to do this section slightly differently than we would a review from a prosumer NAS company. While many consumer NAS devices are fully custom, the FreeNAS Mini XL+ is perhaps best described as “custom off the shelf” as the unit uses a unique mix of components. The iXsystems team did far more than simply slap together a few components.
We have a short overview video of the NAS if you want to listen in while you read along:
We wanted to start this off by showing packaging. Many of these units will ship directly to their end destination. On the other hand, many will ship to an IT office, configured, then sent to a remote branch office. The FreeNAS Mini XL+ packaging is excellent using dense foam to protect both the chassis as well as the drives.
The drives are already installed in their hard drive carriers if you order them with the unit which saves several minutes during setup. Instead of needing a screwdriver, you can just put these drives into the NAS and get going. The box is also designed for re-shipment. For example, the handles are not simply cardboard but have extra handle reinforcements to prevent ripping around the handles. If you want to re-ship the unit to a branch office, this is what you want.
The front of the FreeNAS Mini XL+ is built around an Ablecom chassis. Ablecom is the ODM chassis maker for Supermicro, but it also markets its own products.
The particular chassis the FreeNAS Mini XL+ is using is the same 8-bay unit the original FreeNAS Mini XL used and is fairly hard to source in the US standalone. Actual dimensions of the unit are 8.4” x 11” x 15.1” or 213 x 279 x 384mm if you want to compare it to other units.
One item you will note is the drive trays as we open the unit. We tested and the drive bays are standard Supermicro spec. That means you could use a standard Supermicro 3.5″ drive tray in this chassis. iXsystems is not using simple Supermicro 3.5″ trays. Instead, the company has its own locking trays for the FreeNAS Mini XL+. That is important because it keeps the drives in place even if someone in a remote office brushes against the latch mechanism. Beyond that, the front cover locks which give two layers of protection. That is a big deal for edge storage. This double layer of protection is beyond what a consumer NAS vendor like Synology or QNAP will provide, and even more than many general-purpose OEMs such as Supermicro.
LEDs, power and reset buttons, and USB 3.0/ 2.0 ports are great features behind the cover, but our test unit had an additional feature: a ZIL/ SLOG device. In the optical bay, there is a 2.5″ hot-swap SSD bay. Our unit housed a 2.5″ 480GB ZIL unit that is part of the “Write Cache” package. As a quick note, our pre-production unit did not have this drive over-provisioned to 16GB yet.
That is not the only cache available. Inside, there is an L2ARC read cache drive, also 480GB in size. We were told that the 32TB and 80TB capacity options on Amazon include this L2ARC as standard. As one may expect, the ZIL (write) and L2ARC (read) cache devices help the ZFS-based FreeNAS Mini XL+ increase its write and read performance. You do not need these, but they can offer substantial performance benefits.
Inside, the FreeNAS Mini XL+ is wired for the eight drives, all of the LEDs and buttons, fans and etc. to make everything work. The motherboard is similar to a Supermicro A2SDi-H-TF, but with a twist. The iXsystems team has added an active cooler to the system to ensure reliable operation even in this chassis. We like the use of the mITX form factor motherboard. In theory, if years down the road one wanted to upgrade or repair their system, this would be an easy swap to a newer generation motherboard and CPU.
One can also see a SATADOM for the OS boot image. For expansion, there is a PCIe Gen3 x4 slot for the 10GbE SFP+ option and a M.2 slot that one can use for either a SATA or PCIe Gen3 x2 NVMe SSD.
One of the enormous improvements in this generation is the Intel Atom C3758 SoC. This is an 8 core/ 8 thread CPU with a 2.2GHz clock speed, 16MB of L2 cache and the full set of high-speed I/O lanes (20) that the platform allows. Technically, the SoC also has the mid-range Intel QuickAssist technology.
An enormous benefit is RAM. While the Intel Atom C2000 series that headlined the original FreeNAS Mini line utilized DDR3 unbuffered ECC DIMMs, the Intel Atom C3758 supports DDR4 ECC RDIMMs. Standard, iXsystems includes two 16GB DDR4 RDIMMs in the unit. One can add two more 16GB RDIMMs for 64GB total. Technically, the platform supports up to 4x 64GB DDR4 RDIMMs for up to 256GB of RAM. If you need to run a bunch of redis VMs at the edge, you can have tons of RAM for an 8 core Atom CPU. Likewise, when we see 20TB+ hard drives and this unit scales to 160-200TB of storage, you can add RAM to ensure ZFS has all the RAM is could want.
There is another trick in the FreeNAS Mini XL+ customization. In the front of the chassis, there are two Nidec UltraFlo fans. The standard Ablecom chassis and the sister smaller Supermicro SC721 chassis both are subject to poor airflow because they lack front fans. Here, iXsystems has fans mounted to keep the unit cool. You will also notice that all of the cables are zip-tied to resist being moved into the path of fan blades during shipping and to provide better airflow. This is easy enough to do yourself, but it is nice to not have to, especially in a small chassis like this.
On the subject of cooling, our FreeNAS Mini XL+ had a unique cooler. The rear exhaust fan was translucent green. That is not something we normally see in these chassis and it is another area that has been customized to ensure quiet operation.
The rear I/O here is relatively straightforward. There are two 10Gbase-T Ethernet ports in the middle. To the right is a VGA header. To the left are two USB ports and an out-of-band management port.
To us, this is a big deal. Adding out-of-band management means that the FreeNAS Mini XL+ can be managed like a standard server. Indeed, one gets full functionality like remote iKVM features and power management features. If you are trying to deploy a lower-cost NAS to the edge and manage it centrally, this is a welcome feature over the traditional consumer NAS in-band management.
On the management side, we will next move to the software and management of the FreeNAS Mini XL+ before we get to the performance, power consumption, and noise.
![Pinterest](https://pinterest.com/pin/create/button/?url=https://www.servethehome.com/freenas-mini-xl-plus-review-8-bays-10gbe/&media=https://www.servethehome.com/wp-content/uploads/2019/07/FreeNAS-Mini-XL-Plus-Cover.jpg&description=Our FreeNAS Mini XL+ review shows how this 8x 3.5" + 2x 2.5" NAS performs and how it is uniquely adapted to run FreeNAS at the edge and remotely managed){kind=link}
Maybe I missed it, but I didn’t see the RAIDZ-level configuration.
Well-done review! Extraordinarily thorough. I’ve always thought of these as storage only but you’ve got me thinking about replacing some lightweight VM hosts too.
Misha E. they’ve got RAID-Z on pg3. We use freeNAS and the setup part RAID-Z is very close to RAID-Z2 other than one select box.
Thanks, Stanley.
I wish they’d have a Xeon D option too.
Another great STH review. I’d have liked to have seen compatibility testing with (4) 32gb dimms and (4) 64gb dimms.
Why’d you use 4TB drives not 8 or 10TB drives? I mean if you’re going 8-bay instead of 4-bay wouldn’t you want more capacity than a 4-bay offers? 4x10TB=40TB. 8x4TB=32TB. In your RAIDZ2 you’ve got 20TB usable which is RAID10 like and you’d have 30TB in RAIDZ so more than this 8-bay. You’d also save a load on power and noise with helium 10’s instead of air 4’s.
Supermicro needs that dual fan bracket on their CSE 721
Can anyone tell me what HDD model they use? Thanks
Great review. I just placed an order for a low end TrueNAS X20, maybe I should’ve saved some cash and gotten a pair of FreeNAS Mini XL instead :D
What SuperMicro Board is that running, and if someone has a failed existing freenaz with the asrock board, could they replace it, it’s the same size, but would the backplane be the same ?
@Lukas,
no idea; the configuration/order form on iXSystems online shop does not reveal details about the drives they ship. The review here mentions that the 4 TB drives shipped were some WD Red’s. But then again, since the order page does not disclose any particular drive model that is being shipped, iXSystem might at any time decide to ship different drive models from different manufacturers if they deem those different drives fit for purpose, without you knowing about it until the device is being delivered.
Fortunately, if you don’t want to order a “mystery box”, the FreeNAS Mini’s can also be ordered disk-less (without any drives), leaving it to you to buy drives of your choice from your preferred vendor(s)…
Drive temperatures in that chassi?
What was the RAM configuration? Were there empty slots for upgrading or would you have to replace the sticks that come with the unit?
Will there be an in-depth review of the FreeNAS Mini E as well?
@Lukas, the hard drives are 4 TB WD Reds. Hard drive selection is discussed on page 4 of the review.
@Will I’m looking on P1 and it’s 2x16g sticks and there’s 4 slots so 2 slots open.
ZFS has integrated QuickAssist (QAT) acceleration on some platforms. I am unsure if your version of FreeNAS has that capability. I would be highly interested in seeing the performance of the QAT when QAT-capable versions of ZFS are used. Since it accelerates raidz calculations and compression, this could be very valuable at extending the usable SSD capacity, without sacrificing (much) performance.
We have this in our office. It works flawlessly, my only comment is that those 2 front fans ave quite noisy. I wanted to put this NAS on my desk and I ended up moving it to other room just for the noise.
From a maintenance stand point, I’m curious as to how updates and patching is handled.
While this includes the OS, I’m also wondering about the ipmi and how ixsystems gives us a single point of service.
Background: Supermicro has had it’s share of ipmi security issues and I’ve often found them to lack timely patching and short eol windows. (Ivey Bridge servers are still in production environments, I’d expect ipmi patches to be made available).
Does ixsystems take over from Supermicro?
I have a custom build with the same mainboard Supermicro A2SDi-H-TF and 32GB RAM ECC. Boot drive is a mirrored SSD (Sandisk 120GB). 8x WD Red 8 TB in a RAIDZ2 configuration. I use a Samsung 970 EVO Plus 500GB (onboard NVMe M2 slot) as L2ARC cache and still get only 500-600 MB/s read speed. Then i put a second Samsung 970 EVO Plus 500GB (in PCIe 3.0 x4 slot with Lycom DT-120) as ZIL /SLOG cache and only get 400-550 MB/s write speed. So then i tried to use the second Samsung 970 EVO Plus 500GB as a single drive to get the full speed of the NVMe SSD for video editing over LAN. Even as a single NVME SSD drive the read and write speed are limited to 500-600 MB/s.
I use a Mac Pro 4.1 (flashed to 5.1, PCIe 2.0 x4) with a Sonnet 10G-BaseT PCIe card + Netgear XS505M or the SFP+ approach with MikroTik CRS305 + Sonnet 10G SFP+ PCIe card. Always 500-600 MB/s.
With 8 WD Red (5.400 rpm) in RAIDZ2 configuration i was expecting at least 800-900 MB/s read speed and 700-800 MB/s write speed. Adding NVMe SSD as L2ARC and ZIL / SLOG cache did not boost the performance to saturate the 10 GbE connection.
With iperf3 performance test in both direction i get around 9 Gbit/s. For the read and write speed test of SMB, AFP and NFS shares and iSCSI volume i use BlackMagic Speed Test.
If anyone has any suggestions would be nice to hear.
Zoran,
can you get intel optane 900p drive instead of samsung?
Would be better, than samsung.
I have one installed as zil device, perfect, the 10GBE can be fully saturated by smb.
Patrick,
i would like to get this ablecom cs t80 case to upgrade my 4 bay rig. :-)
I think supermicro and ixsystems gets chassis from the same manufacturer, from ablecom.
The 4 bay case of supermicro is exactly same, than 4 bay freenas case of ixsystems.
I am considering purchasing this computer, however, I am looking forward to running TrueNAS Core running on Debian.
Does anyone know if this computer will support a switch to the new software?