iXsystems Launches TrueNAS M60 High-Availability Storage Appliance

For those that use FreeNAS or TrueNAS Core in their labs, and want to move up to a supported and higher-performance solution, the new iXsystems TrueNAS M60 offers something different. The new solution offers significantly higher performance and many new features.

iXsystems TrueNAS M60 Overview

The core of the solution is a high-availability controller with 24x HDDs or 28 SSDs in a 4U system. Beyond there, one can add additional disk shelves to increase capacity.

One of the biggest claims to fame with the new generation is the ability to move a lot more data. Indeed, the new solution is designed to handle over 20GB/s of performance and over 1 million IOPS. We will note that we have already reviewed single SSDs such as the Kioxia CM6 that can hit over 1 million IOPS, however, those are individual drives not in the context of a full system. Also, we are going to note that these performance claims are using TrueNAS 12.0 although the systems will likely ship initially with TrueNAS 11.3 which offers significantly lower performance.

There are a couple of interesting hardware features that we wanted to discuss. The solution now utilizes 100GbE. You may see 64-cores. This is not due to using something like an AMD EPYC 7702P or an AMD EPYC 7502P in each controller. Instead this is two dual-socket Intel Xeon nodes. We asked, and specifically, iXsystems is using the Intel Xeon Gold 6226R chips we reviewed.

Many of our readers may question why not use AMD EPYC here. Doing so would eliminate the UPI constraint. Specifically, the 2nd Gen Intel Xeon Scalable Refresh SKUs only offer a 2x UPI link configuration, not the higher-end 3x UPI link configuration that one finds in higher-end solutions such as Nutanix nodes. It seems iXsystems is OK losing a third of the NUMA-node to NUMA-node traversal bandwidth and does not want to use AMD EPYC where this hop would be over the same IO die with Rome.

Getting 16 cores with the Xeon Gold 6226R is inexpensive. The company gets a few benefits. These include the fact that Intel Xeon platforms generally have more form factors available and better NVDIMM support. Also, iXsystems is using a non-transparent bridge between the two nodes for high-availability, so apparently the demand on that bridge does not benefit from the speed and latency advantages of PCIe Gen4. Intel has a long history offering NTB functionality.

When asked why not use AMD EPYC 7002 that has more memory bandwidth, more PCIe bandwidth, and can potentially eliminate the very slow socket-to-socket transfers At STH, we were told that:

“TrueNAS 12.0 does have better AMD support, but we are only just starting to evaluate the performance. Memory bandwidth and latency are very important to ZFS.” (Source: iXsystems statement to STH.)

We think this is very important. A key takeaway for STH readers is that iXsystems is still working to support AMD solutions. For those thinking of building AMD-based TrueNAS Core systems, remember that iXsystems is still using Intel Xeon. Part of that is also due to legacy FreeBSD which has been a bit behind Linux in terms of AMD EPYC 7002 support. Linux has a much larger ecosystem and has had significantly faster cycles of enhancements for the new CPUs. TrueNAS Scale will be the company’s move to the Linux space where it will get better hardware support which may change this equation in the future. This seems to be a case where the Intel Xeon was the safe choice, which makes sense in the storage world.

Final Words

We covered the initial iXsystems TrueNAS M-Series Launch years ago. It is great to see the model lineup expand and evolve. We are looking forward to TrueNAS 12.0’s release. For those looking for a middle ground between a TrueNAS M60 and a build-your-own solution, we reviewed the FreeNAS Mini XL Plus last year and plan to circle back and cover new solutions in that space in the near future.