This is the release we have been waiting for. Today, TrueNAS Scale has finally hit release. We are just going to quickly discuss some of the big points but will have more content on STH for the release at a later date. We wanted to get into the release a bit, but then give a perspective on this going forward. We know our readers will likely test Scale, but wait to deploy, and that is reasonable at this juncture. Still, there is a roadmap so we need to also look at where this is going.
TrueNAS Scale 22.02 Released
First off, let us start with what is TrueNAS Scale, and where are we in the process. TrueNAS Scale is based on Linux instead of FreeBSD like TrueNAS. We get all of the storage features we would expect in a modern solution like snapshotting, replication, iSCSI, SMB, NFS, and S3-compatible object storage. It then layers new features and Linux features on top of that new base like KVM for virtualization, Kubernetes support, and a Gluster-based scale-out storage solution. Gluster is run across ZFS and that allows for having many benefits of ZFS like built-in compression, snapshotting, and data integrity with the scale-out multi-chassis support of Gluster.
Today marks the Angelfish release for TrueNAS Scale 22.02. This is the release that is stable enough to really start using.
Next comes the future updates, and versions. iXsystems says TrueNAS Scale after a few updates should be ready for more critical deployments. Also, there are a lot of features like Gluster-wide snapshotting, FIPS 140-3, Wireguard, and more that are coming with the Bluefin version later this year.
A quick note here is that iXsystems is not discontinuing TrueNAS (non-Scale.) Instead, that will go to TrueNAS 13 in the near future as the company’s scale-up solution. Bascially, without directly putting it on the slide they are saying if you want reliable storage, continue to use TrueNAS for now. At some point in the future, we can see iXsystems doing another rationalization effort and migrating customers to TrueNAS Scale, but that is likely not a 2022 or probably a 2023 activity. One will also have the option of using TrueNAS Scale nodes for application hosting and TrueNAS classic for storage, basically replacing VMware, Red Hat, Nutanix, Microsoft, or other VM hosting platforms with Scale.
Logging in, if you have used a recent TrueNAS installation, Scale will look very familiar. If you are still using an ancient FreeNAS version, then this will feel much more modern.
At the heart of TrueNAS Scale, it is still a storage-led appliance. After basic permissions and networking is setup, the next step is that you need to create a pool. This is similar to how you would interact with a QNAP or Synology NAS, also NAS-led solutions, but is a bit different than VMware, Proxmox VE, and others. As a simple example, with Proxmox VE, you can install root onto a mirrored ZFS array and then run VMs directly from that (or even a single disk.) With TrueNAS Scale, you start with a storage pool.
The big features aside from the scale-out are the KVM virtual machine and containers. KVM is probably the #1 hypervisor by a long shot since it is used by major cloud platforms. TrueNAS Scale has a GUI for it, although the layout is very vertically focused.
Likewise, one can setup Kubernetes and start running containers.
TrueNAS Scale also has an application catalog that one can use to install apps easily, similar to what you would see on a QNAP or Synology.
The trick with all of this, of course, is the scale-out capability. By building on Gluster, one can have storage available to many nodes. This allows migration of workloads to happen across a cluster and iXsystems can have container and VM storage on Gluster. That also means that adding more compute, memory, and storage can happen simply by adding additional nodes.
TrueNAS Scale v. Proxmox VE
After having played with TrueNAS Scale during development and down to the release date, and also having used Proxmox VE since 2013 (we ran STH on Proxmox VE in 2013 in our first little colocation installation) I think I have a perspective on the two.
The two solutions may seem similar, but they are actually quite different. TrueNAS Scale feels like TrueNAS Core, but with better virtualization and Kubernetes support. It feels like it grew up as a NAS and then added features more in-line with some of what Synology and QNAP offer, with a critical difference: there is the promise of scale-out using Gluster and ZFS. Proxmox VE feels more like a team that wanted to displace VMware directly. Under the hood, Proxmox VE also feels very much like Debian Linux, you can apt install packages for example. For now, it is really the difference between roots in a NAS environment or a virtualization environment.
With this release, as mentioned earlier, TrueNAS Scale is OK for smaller installations, but higher-end installations are still a few months away. Proxmox VE we have had 100 physical node clusters running in the lab, and have had those for years. We also have a number of production and lab clusters set up.
One of those Proxmox VE clusters, a small lab cluster running Ceph on all HPE ProLiant machines just failed due to multiple failures in the servers. We were going to have the story of that one in January but it keeps getting pushed out. Even with that, and having had clusters fail before, updating clusters (a bit more involved with Ceph), and so forth, Proxmox VE is still our recommendation if you needed a SMB VMware alternative today. You can purchase relatively inexpensive support and everything works.
That is today. Where TrueNAS Scale is headed is a converged scale-out infrastructure with a single pane of glass management in TrueCommand. Two years from now, if Proxmox VE does not up its game on storage management via the GUI (one can do everything with the CLI, and it has been adding features to the GUI slowly) then the default recommendation will be TrueNAS Scale instead.
Final Words
Last week I dug out an old discussion document from September 2016 where I headed over to iXsystems and voiced that they needed a scale-out hyper-converged (then FreeNAS) option to take on Proxmox VE. It has taken a long time, but TrueNAS Scale is the answer.
What is really cool here, is that iXsystems is about a year away from an endgame-type product for those in the SMB market. There are so many SMB/ SME customers that use Red Hat/ VMware and spend a ton on it just to do some simple virtualization or container orchestration. For those companies that also spend large amounts on storage, only to have to make huge up-front investments in capacity, scale-out allows compute, memory, and storage to be added over time in smaller chunks.
As much as I think the Proxmox team should be nervous about TrueNAS Scale, QNAP, Synology, and VMware may need to be doubly nervous. QNAP and Synology should have had a great cluster solution out by now, but have not. Synology’s hardware is at a huge premium and is usually limited to older hardware. QNAP has more modern hardware. Both will come under a lot of pressure with a TrueNAS Scale solution that can run on a wide array of hardware and can both scale up and out. Paying vast sums of money for Red Hat/ VMware solutions once this matures is going to seem fool-ish at current pricing.
Make no mistake, this is a huge release in the world of SMB IT, even if it takes some time to propagate and mature. We are extraordinarily excited about TrueNAS Scale at STH and will be using it more in the coming months.
Any plans to do a comparison between TrueNAS Scale and Proxmox for tiny clusters? I’ve got a three-node Proxmox cluster hosting a few VMs on HP Thinclients. Curious to try out Scale in its place, see which solution would make more sense. This is of course knowingly using a non-recommended setup, mostly that I only have a single boot SSD (16GB) and a single (256GB) data drive.
I really want to read about the cluster failure and how you resolved around the issues. Those type of stories are great reads
My guess is that we will have some more comparisons in the next few months. We are quite backed up with content right now.
Churchill – the piece was written a month ago, but has not found its way to the publishing queue.
Lawrence systems did a performance comparison between RC1 od Scale vs Core, and the performance of RC1 Scale was atrocious. He will do another comparison soon of the final release from today of Scale. This is going to be very important to watch.
Don’t know. Gluster is not great in terms of data integrity and stability. We had countless problems with it ourselves (granted, we did use XFS instead of ZFS). Will be interesting to see how this plays out in the real world.
Has there ever been a reason posted about why they switched away from FreeBSD?
B5Geek – they still support FreeBSD in TrueNAS Core. If you go back to some of our older articles, our thought is basically that once OpenZFS used Linux as the base, it makes sense to take advantage of the much larger ecosystem.
I am so happy to see VMware and all these big overpriced monsters have some strong competition. Now the question will big enterprise ever adopt this.
It does say this is for smb but same time zfs, Gluster, and lots of other “cloud”, enterprise features and this product will mature quickly.
Do you think customers will go to this instead of VMware, red hat, or stick with that?
I know more and more big enterprise are running to the cloud or using VMware on AWS cloud.
As someone who just helped a SMB getting Proxmox VE setup on hand-me-down R720’s, I’m instantly pondering if I’ll being doing an overhaul in the next 12 months.
Patrick, I think an excellent video / tutorial would me migrating VMs from Proxmox VE to TrueNAS Scale and back, so folks can get comfortable with this process. KVM is KVM, right?
I had a VMware server in my house for lab and ‘production’ (DNS etc) purposes. I wanted to switch to a software raid solution (don’t want to pay for a new controller if mine would fail), and have an easy way to run docker containers. TrueNAS Scale seemed like a great option, so I installed RC2 last month. Unfortunately, after a couple of days trying to figure out file permissions and their issues (ISO uploaded via SMB couldn’t get mounted in KVM. Their forum couldn’t solve it either), I gave up. This was after days of trying to get everything to work via Linux shell, as a non-Linux guy.
Perhaps TrueNAS can work well if you have a lot of experience with it and Linux. But VMware is just a lot more fool-proof. And since labor is expensive, the total cost of ownership of a VMware platform might still be worth it.
FYI I landed on Unraid for my home server and so far I’m very happy. Worked out of the box, not a single issue while deploying my use cases.
While I’m happy that Scale is out, I’m using it for Minecraft for my kids. True Charts hasn’t gotten MineOS working that I know of as of yet from last looking. I’ll go research it here after posting this. But for my use case rn it is working and I’m not moving anything over until I know for sure it is working. The old moto of “if it ain’t broke, don’t fix it” screaming in my head. lol
I’m confused at the Proxmox comparison; its completely irrelevant.. It’s like you ran out of things to talk about and decided to compare a NAS to a Hypervisor? Was there any question that Proxmox as a NAS? at any time? really?
If you had compared TrueNAS to Synology.. I would be far, far, less confused.. the only comparison is their use of OpenZFS.. thats it..
Repeat after me.. one is a NAS, the other is a Hypervisor..
I wouldn’t even use ZFS normally under business conditions with Proxmox.. I would use CEPH. Ceph is a distributed object store and file system. As a HA drive backplane; designed for clustering nodes together that is far more suited for a 3+ Hypervisor node / cluster setup..
Ceph, designed by Red Hate, IBM and the likes is a very competitive solution. You should read up more on CEPH at ceph dot com. You might sound more competent next time. ;)
Harsh comment and a bit unfair too. Isn’t the whole point here, and as explained in the article as well, that TrueNAS Scale is moving from NAS to hyperconverged infrastructure which aims to consolidate NAS and hypervisor in one? And in that regard the comparison with a hypervisor is perfectly valid. So maybe be a bit more careful with calling other people incompetent. ;-)
I certainly share the picture Patrick is painting here. But as he said, it will take years to get there.
Weeks ago i tried to converge all my NAS and VM on a single SCALE system. It did not go well.
Performance issues, missing USB passthrough for VMs, etc.
For VMs i am back to Proxmox now, and SCALE is used for storage. Great setup for now, and i will test SCALE again in 2023/2024 for VMs. But right now, it is not there yet.
Still a huge potential to make a positive shift in the market!
And iXSystems is a great company, the development is very open and i have been impatiently alpha and beta testing for a year now. Cant wait for the day when TrueNAS SCALE can be my All-In-One solution!