Some Thoughts on Customization
Working on this one, we already did a lot of customization similar to the fanless unit. One customization we did was to add the SFF-8087 to SFF-8088 cable and a QNAP TL-D400S 4-Bay SATA JBOD DAS enclosure. While that was interesting, we got a second cable for a second system and had two systems using the QNAP TL-D800S 8-Bay SATA JBOD DAS. An important note, if you do this, is that the two systems cannot access all eight drives as you would expect from an enterprise SAS enclosure. As a SATA enclosure, the drives can also only be attached to a single head node. Instead, you would get 4 drives to one and 4 drives to the other. The benefit is that the 8-bay costs less on a dollar-per-drive basis than getting two 4-bays. A negative is that all drives would be in a single enclosure.
Internally, something that you may want to try is to find a second SATA power cable if you are going that route. A decent idea is to ask when you order for one as an add-on. You would want to use lower-power SATA SSDs, but since we had two units, we tried using two SATA SSDs internally using the SATA data and power cable from both units in one of the systems and it worked. Having four internal drives is a decent storage feature as well.
If you want to try some of the colored DACs that we use, those are fun just to try out and they help keep things organized. Otherwise, we used fairly cheap SFP+ SR modules.
In terms of RAM, technically, the Atom C3758R will support 256GB in 4x 64GB configurations. In this system, there are only two slots. With the 8-core CPU, 2x 32GB feels like a lot, and we would probably get 2x 16GB instead, deploying at 4GB/ core or 4GB/ thread.
Key Lessons Learned
The Intel Atom C3758R is an interesting chip. Whenever we review these Atom C3000 series parts, we hear that they are too slow to be a homelab node, or that they do not have enough performance for storage.
Neither is really true. Netgate, for example, can push over 18Gbps of traffic through the same CPU and NIC setup in the Netgate 8200. Using TNSR, Netgate can get over 15Gbps on IPsec VPN and over 18Gbps with a firewall’s ACL rules using similar hardware. Achieving over 10Gbps on a $300 Qotom network device ($400 configured) is phenomenal.
Having the SFF-8087 port on the back is a bit strange. Still, combined with an external 4-bay enclosure, it has a slightly faster processor than the TrueNAS Mini X+.
In either case, as a firewall and/or NAS device, this system has plenty of performance. Perhaps the biggest testament to that is that the companies that make appliances for popular software use a similar CPU.
We now have 5+ of these units (three fanless, two rackmounts), and they have been running well for months. Something like the Minisforum MS-01 we reviewed is a higher power and higher-performance option, but this has an ECC memory option, more networking, and built-in connectivity to a 4-bay DAS. If you want to go bigger, the MS-01 is a great option, but there is going to be a huge portion of the community that will want either a fanless or a rackmount option, and this Qotom has both options and a lower price point.
Final Words
The Qotom box is not perfect. For example, the 2.5GbE and 10GbE ports use strange enumeration, making the ordering strange when looking at the box. It feels like a small fan inside would alleviate any concerns about heat, although we did not run into any. The top of the case requires some work to align since it is not machined like a fine Swiss watch. At some point, however, these are fairly minor points because of the price. Four 10GbE ports and five 2.5GbE ports with an 8-core CPU and more for around $300 is a great deal. There were even small upside surprises like the Mean Well PSU.
We have two other 1U Atom C3000 series boxes that are slightly more expensive, and slightly less feature-rich that might be close to this one, but it is hard to argue that this is great value. To me, the biggest competition for something like this is probably the fanless version of this system. I think a lot of folks would prefer the fanless version. At the same time, if you want a rackmount 1U Intel Atom C3758R appliance, then this is a great option.
Where to Buy
If you want to check pricing or buy something like this, here are some affiliate links to listings.
- 1U AliExpress (Affiliate Link)
- 1U Amazon (Affiliate Link)
- Fanless AliExpress (Affiliate Link)
- Fanless Amazon (Affiliate Link)
Prices change regularly.
That’s an amazing configuration, and the ECC option was critical to include IMO. I have three Supermicro boxes configured as a 3-node cluster. Each include multiple SFP+ and Intel i350 ports and support ECC. I upgraded all of them with 128GB RAM, three NVMe’s, and a dual port 25GbE Mellanox card.
This is why I think the MINISFORUM MS-01 falls short of practical deployments in a homelab, because without ECC support, the true potential cannot be realized. The MS-01 is a great machine, but was a missed opportunity—hopefully the MS-02 includes ECC.
Everything in 1 box eh? A hacker’s dream come true!
Thanks for the great job STH team is doing
As you mentioned the NVMe area gets really hot that is why it is important to use low performance NVMe to manage heat. In my Mini PC I mounted 3x80mm FANs & replaced the bottom cover with a mesh just to keep that area relatively cold. I think this 1U setup it is great but it would be better if they made a mesh to mount tow fans instead of the WiFi antenna holes
Again thanks a million
My fanless non-rackmount model came with a Channel Well Technology (CWT) PSU, so even those seemingly no longer come with no-name PSUs, at least not all the time.
A rackmounted box, where the power switch is at the back of the box. That will work just fine.
What sfp to rj45 modules can be used with one of these??,,,
Thanks
My only problem with this is that the new E-cores are twice as fast as these. It’s a hard value prop when the N305 is more than twice as fast and uses less power. It feels like there will imminently be a new C-series line and that any C3000 series is already obsolete.
Granted, that’s somewhat hyperbolic, I do understand the differences between the N305 and the Cx000 atom lines (I have 3x C3758 systems in my lab today).
But I hope you see my point; the C3000 series is getting quite old to be part of greenfield deployments. Unless you need the crypto throughput or >2.5G networking, I’d add an N305-based system to my lab before any C3000-based systems.
How is the power supply connected to the motherboard? All the pictures show the wires disappearing under the board, but I do not see any connector in the picture of the underside.
page 3: “Mean Well LRS-50-12 a 12W 50V unit”
Should that be a “50W 12V unit”?
I have the fanless version, installed 32GB ECC RAM and also noticed the long boot time. I’m wondering if they ever disabled a lot of extra memory training/margining tests that developers usually use before release. It doesn’t hurt to have all that extra training enabled, but the value is debatable in a production system.
I would like to see also Wireguard perfomance tests.
To my knowledge it relies purely on CPU power so that would be a nice datapoint.
How does the 1U unit handle thermally vs the fanless “desktop” unit?
I wouldn’t buy any C3xxx based system at this point. Too slow vs Nxxx options.
Why would you have a fan on the bottom of a rack mount unit? It will be blocked by whatever is mounted below.
In the review the PSU specs are mixed. It’s 12V and 50W, not 12W and 50V
TBH, ECC is pointless for networking usage, thus I’d rather have more horsepower of MS01.
How about putting a m2 to sff8087 adapter in there as well? Then you could connect to that 8 drive external housing and access all drives
Does the AST2400.firmware in this provide KVM over IP?
Has anyone installed VyOS or what do you use for routing on these? Also some people complained about the CPU being slow compared to other options… are any of these other options available in rackmount or with 10GbE?
Does the ASPEED AST2400 BMC chip provide an actual IPMI functiion? Or is there vPro/AMP on the system?
@Jason Nxxx processors don’t have Intel QAT like the C3xxx processors do, so they’re not as good for routing or encryption.
I see a C3808 12 core processor option. But on the 12 core version, its has SFP vs SFP+ ports. Seems odd, anyone order the C3808 version?
I’m not getting 5gbps symmetrical with my box. It’s less than half that with no scanning on for Untangle and S+RJ10 modules.
Ended up going with a protectli vault 6.
I have just purchased the C3808 version. I contacted Qotom directly and they confirmed to me the 4x ports are indeed 10G SFP+. I’ll get back if somehow that is not the case once I have received the item.
If you are looking for an additional SATA cable, I believe this will get you pointed in the right direction: “PH2.0 Small 4Pin Combo 7+15 Pin”. (I have not received mine yet to verify.)
Hey, thanks for the review! I’m considering migrating my current pfsense setup from a old server to one that has SPF and higher throughput because they are starting to install 10Gbit fiber around here.
I have a question about this board (or boards with SPF in general). My current setup involves a ONT that outputs 1Gbps via Ethernet and a router with 4x1Gbps Eth ports (1 for WAN). Can I use one of the SPF+ ports for WAN and avoid having another piece of hardware to do the conversion? That way I can just plug the fiber cable to this board and use the SPF ports to connect to my switch (which has SPF up to 10Gbit and 2.5 Ethernet ports). I have been searching for information about such a setup but haven’t found anything and I don’t want to risk the buy before being sure I can do the setup I want.
Thank you!
I received my PH2.0 small 4-pin cable with integrated data wires from AliExpress and was disappointed to find that the SATA data plug is a right-angle connector that is bent in the wrong direction and there is really no way that it fits on this motherboard. It is shown accurately in the pictures but it didn’t even cross my mind that this would be an issue. So I have now ordered a separate power-only cable.
In the video, a bug regards connecting the SFP port to a switch using a Debian OS is mentioned.
I would want to connect two of the SFP ports to my switch, one for the LAN side interface and one for the Proxmox management interface. Given Proxmox is on Debian, my understanding is this is not possible (although maybe passthrough of one to OPNSense would allow the LAN connection).
Can anyone advise on this – have people succesfully used the SFP ports connected to a switch with Proxmox and/or hardware passthru with OPNSense?