Netgate SG-5100 Performance
Normally at STH we have used straight iperf3 which gives a good idea in terms of a maximum throughput in a simple use case. Instead of using the simpler iperf3 traffic pattern, we have been testing the Netgate SG-5100 with an IMIX set comprised of the following:
- Packet size: 60, pps: 28
- Packet size: 590, pps 16
- Packet size: 1514, pps: 4
Here we are using the IMIX above to push 1GbE line rate from the test system through the two Netgate SG-5100 systems. The net result is that we are pushing well over 300K packets per second along with the 1Gbps of bandwidth.
That seemed interesting, but we wanted to get more expansive. Note here that we are using DPDK on CentOS 8 on the test systems and pfSense for the SG-5100’s network OS. We are also using IX0, IX1, IX2, and IX3 as those correspond to the 1GbE ports from the Atom C3558 SoC. We are using these ports for traffic and assuming the other two Intel i210 NIC ports are being used for management.
As you can see, pushing traffic through all four ports at 100mbps gave us 400Mbps and over 130K packets per second. We also did not see any packets being dropped on that run so we doubled the speed to 200Mbps per port.
You can see here that the L1 traffic is a bit higher due to overheads showing over 840Mbps. Still, we are not getting dropped packets so we are moving up another 100Mbps to 300Mbps here:
Here we hit 1.2Gbps combined with around 400K packets per second using our IMIX. Again we pushed, this time to 350Mbps per port.
Here we hit 1.4Gbps passing through both firewalls and over 460K packets per second. When we pushed to 4x 400Mbps streams, we started to see dropped packets:
Each of the two Netgate SG-5100s has four ports (2 LAN and 2 WAN) that are routing traffic across. This is likely more than a typical setup would see in terms of sustained usage. We dialed back and found that 4x 360Mbps was not dropping packets at around 475K packets per second and 1.43Gbps.
Since we assume that not everyone will have four ports going, we instead are pushing packets through two ports through the solution. Here we tried two 650Mbps links and saw the setup working without issue:
When we pushed to around the same as we saw on four ports with 700Mbps on two ports, we started to see packet loss again.
If you have a single Gigabit WAN connection, it is likely that this setup is fine. If you need to run multiple WAN connections, this solution seems to be able to hit over 1.3-1.43Gbps without issue.
This is all great. However, remember that pfSense is considered the lower-performing but easier to use network solution. In our next piece, we are going to show TNSR numbers, as well as showing IPsec VPN performance across the two nodes and two network operating systems in this quad-port configuration.
Netgate SG-5100 Power Consumption and Noise
We used our pair of Extech TrueRMS Power Analyzer 380803 units to take measurements at different points of the Netgate SG-5100 usage. Embedded platforms tend to spend more time at the edge in offices rather than in higher power data centers, hence why we do our testing at a lower 120V voltage. Here are the figures:
- Lowest idle: 12.1W
- Normal idle: 13.8W
- Maximum observed: 19.2W
Overall, these are great numbers. One benefit to not including a traditional ASPEED BMC is that power consumption is 4-5W lower than BMC enabled platforms.
Now for the easy one, noise. There is none. This is a completely passive device which means no fans and no noise.
Final Words
These units are priced at $699 + $19.99 for an optional wall mount kit and community support. There is a $70 off holiday special. If we are being transparent, one can build a Supermicro Atom C3000 based solution for $400-500. The Supermicro solution will have a BMC but will use more power.
There are some that will see a $150-250 premium as completely excessive. Those people are firmly in the build-your-own or get some inexpensive network appliance with a less than a well-known supply chain. That is fine too, this product is not for that segment.
Netgate also offers optional support packages with the Netgate SG-5100 which are must-haves for certain business purchases. The company also maintains security patches and updates to pfSense and TNSR for its products which may not happen on other platforms. There are others that simply like the idea of supported open-source software but who simply want to have a box delivered rather than piecing together a system. For all of these markets, the SG-5100 is ideal.
Next up for the SG-5100 pair is an investigation into TNSR versus pfSense with a specific look at IPsec performance. Many have noticed that IPsec VPN experiences with pfSense is good, but it is not the fastest. TNSR aims to fix that and the SG-5100 is perhaps the starting point for those that want to deploy that software. Stay tuned for more on STH.
No SFP+ port on this device means you’ll have to factor in additional costs and space for a media converter when using fiber. Too bad for what’s otherwise a great little device.
$700 is not the cost of the device. The cost is $700 and whatever subscription you’re using at work for support.
It’s nice to see that ya’ll are doing pfSense gear again.
> If we are being transparent, one can build a Supermicro Atom C3000 based solution for $400-500
for example?
Do you have a reference or documentation to your test methodology? I would like to replicate the results.
Would like to see alternative to my apu2c4, Iam running proxmox (pfsense as fw,router and openwrt as dumb ap,802.11n and 802.11ac), its working ok, but startup takes 5mins.
I want something (mini itx or smaller) more powerfull, aesni, iommu, nvme + 2x m.2 for two wifi cards..
@alekun
eg https://www.wiredzone.com/shop/product/41113201-supermicro-sys-e200-9a-compact-embedded-intel-processor-iot-barebone-6982
Excellent write-up! I’ve been extremely curious about TNSR since hearing of it’s existence quite some time ago. Very little information exists on it so I eagerly await that portion of the review!
How about SG-5100 pairs for HA?
Did the SG-5100 review using TNSR ever get published?
This “Netgate” SG-5100 looks, to me, to be a rebranded / relabeled device designed and manufactured by “Lanner” (look ’em up).
I’ve got a (now several years old) “RouterMaxx 1106” embedded device acting as my firewall / router that was also manufactured by Lanner. It was originally sold with RouterOS (and runs OpenBSD and FreeBSD wonderfully!) that looks *very* similar to this SG-5100 — Atom CPU / SOC, SODIMM slots for RAM (upgradable!), 6 x Intel 1 GbE ports (via two separate MACs), serial console, metal heat-sink for a case, exact same style of power connector and reset button, roughly the same price, and so on…
Google “RouterMaxx 1106” and compare images of it to this device and you’ll see what I’m talking about. I wouldn’t be surprised if Netgate is getting many of their devices from Lanner — and they definitely aren’t the only ones.
I don’t think Lanner sells directly to consumers but you can probably find this exact same device with some other company’s name on it (instead of Netgate) and get it for a bit cheaper.
All that said, I can’t really complain about the device I’ve got. It’s been in constant use for probably ~6 years now (with the RAM upgraded, the CompactFlash card replaced with an SSD, and RouterOS replaced with — at the moment — OPNsense; Debian, FreeBSD, and OpenBSD all “just work” too!) and I’ve yet to experience any issues with it, FWIW.