Virtualizing a Firewall and VPN with Proxmox VE
As one would expect, first we installed pfSense and OPNsense on bare metal and this worked exactly as one would expect. As we saw last time, you will want newer versions such as pfSense 2.6.0-RELEASE that support the Intel i225. Intel was not fast to support the i225 on FreeBSD so the Netgate folks did some work to get it into FreeBSD. That is why support lagged a bit since Intel was focused on Windows and Linux drivers. Performance both in physical and virtualized formats was over 2.1Gbps using a lightweight set of firewall rules and doing NAT from WAN to LAN. There is certainly more that can be done to get more performance, but out-of-the-box for basic setups it is very good.
The big change is that this time we virtualized the solution. These units allow one to use pass-through to virtualize the firewall appliance. We did an entire guide How to Pass-through PCIe NICs with Proxmox VE on Intel and AMD and that was using this unit. One quick note is that some may say 16GB is too much, but as one can see from the OPNsense VM here, having more than 8GB on a virtualized firewall can be helpful if you want to run more.
Here is a shot of the Intel i225-V’s being selectable to pass-through to the VM.
While one can have both pfSense and OPNsense installed with pass-through NICs, for example, if using this as a test appliance, there is a small catch. Only one can be run at a time because otherwise there is a conflict on passing through the NICs.
One major advantage of virtualizing the firewall in this way is the ability to take snapshots. Not only do we get fast VM reboots after a firmware upgrade, but if one has a bad firewall configuration, or a failed upgrade (not as common as it was many years ago), then reverting to a snapshot usually takes only a few seconds.
One of the other fun use cases we have, and we show it a bit more in the video, is that you can add more to this unit than just the firewall/ VPN appliance. For example, we use pfSense and the HAproxy add-on to do SSL termination and be a reverse proxy for Guacamole. Guacamole allows for browser-based remote desktop and terminal experiences, so we use Guacamole to access Project TinyMiniMicro nodes via a web browser. One can also add things like WiFi controllers on these little boxes as well, but we will highly suggest going to 16GB of memory if you want to run more VMs.
Over the years, I have run virtualized as well as physical firewalls. I have been on physical firewalls (mostly) for the last few years. As hardware has gotten better, especially for labs, a virtualized solution can make a lot of sense.
Final Words
Overall, this is a quick pre-holiday weekend piece to show off an update on these small fanless quad 2.5GbE units. This Topton unit has a more robust heatsink, but the Hunsn unit arrived much faster because we purchased it via Amazon versus AliExpress.
This year we are seeing more 2.5GbE devices, and having these little low-power silent devices helps bring another segment into the 2.5GbE era.
For the record, we would recommend this unit over the Hunsun unit, if you prefer the chassis and can wait longer. They are similar internals and there is a case to be made to get barebones and then focusing on better-known brand storage and memory.
Throughput with ips?
For a tech site, this ‚review‘ is extremely light on technical data or details e.g. can it actually handle 4 network ports with data flowing through at 2.5Gbit/s rate, while inspecting and routing those packets the way a firewall is supposed to?
And how much is its performance affected by the firewall running as a VM?
I can’t pick up a clear indication as to if a) network ports connect to the host and data is passed through vNICs, b) network ports are passed through to the VM via VT-d, c) SR-IOV available for a performance/flexibility middle-ground?
Surly these options have a performance impact the readers of this site would like to know, before you just assign a rating, based on what exactly: a successful software installation?
The lack of a second DIMM slot doesn’t surprise me. Even if RAM on Atoms (Broadwell to GLR at least), is in theory supported with two channels, in practice there is no measurable benefit for bandwidth when adding a second module. I run my J5005s with 32GB of DDR4-2400 in two modules, but only for capacity (a single 32GB SO-DIMM fails), not because it helps in terms of performance, which is at a miserly 16.5GB/s for multicore, 10GB/s single. I’m afraid Jasper Lake is the same, but I only ever got a box with a single SO-DIMM slot into my hands.
While Atom’s have 6 PCIe [2.0 with 2.5MT/s each] lanes, you can’t have more than a 4-way split. So with 4 PCIe ports likely going to Intel NICs, that leaves only SATA for storage, which should be ok on a firewall and would still be able to saturate a 2.5Gbit Ethernet link—if there isn’t too much else to do. You do (wonderful) bus diagrams normally, so it’s odd you’re skipping it here.
I’ve used Atoms with pfSense but once I started to put Suricata/Snort rule packs on them, they started to throttle even a 250MB/s uplink. Adding VPN and CPU based (no QAT) based encryption would be rather normal usage scenarios for a firewall, but in my estimate overwhelm this hardware, even more so if you start virtualizing it or split different network sanitizing functions into distinct VMs.
There are ARM firewalls coming out these days with quad 2.5Gbit ports on an even lower energy footprint, but to my understanding they employ acceleration blocks to do the filtering and crypto stuff and use the CPU mostly for orchestration/interaction. And I’m not sure they’ll run with open-source firewalls like pfSense for lack of driver support.
Readers of this site might start hitting the buy buttons on this hardware after your first two pieces, expecting that it will sustain firewalled traffic flow through at near 2.5Gbit/s. But I’m afraid you haven’t gotten to the point of proving that just yet and you should mention that already, and then perform the load test as soon as possible.
@Thomas Hobard
Did you not read the following:
“Overall, this is a quick pre-holiday weekend piece to show off an update on these small fanless quad 2.5GbE units.”
Concur. I’m in the market for a small firewall appliance running an OpenVPN server and Suricata. So difficult to find hard data on what kind of throughput I can expect on these things. Would love to see a follow up!
This is the unit I have running my network currently App ideas
SDN controller (omada/unifi) in a lxc container (have this one)
Asterisk (working on this one) would love to see a sip multiplexing howto with 3cx or similar for ‘how to turn your sip homephone into a bunch of extensions)
freeipa
a better ip managment because pfsense doesn’t do non-connected vlans with l3 switching.
@Pheckphul
The last review of the, “Inexpensive 4x 2.5GbE Fanless Router Firewall Box” was equally empty of any real analysis while both pieces of hardware were given an over 9 score in rating.
Granted a 9.something on this site doesn’t really mean all that much when you see that pretty much everything hits those scores. And yeah yeah, only good products are reviewed so obviously they get good scores, blah blah blah. Please. The level of non-objectiveness and childish levels of excitement on anything related to Intel and Nvidia and lesser degree AMD and most of the big tech companies also makes one have to question objectivity. To hear him talk about sapphire rapids and H100 you would think all disease has been cured, poverty abolished and all nations have done away with war.
On the other hand I am so tired of the writing style overusing the “one” pronoun.
“One can see”, “as one would expect”, etc.
Just count them, for example on page three.
Please stop it, use it sparingly. It’s jarring.
Why would you virtualize and not run directly on box?
Very interesting in that FreedomFi’s Helium miner has a very similar looking chassis, except that the VGA port is replaced with an RJ-45 console port. I wasn’t able to find this on Topton’s site, so maybe they are using a customized SKU.
Could you please clarify memory configuraton. There is 16GB DIMM inserted, a screenshot shows 8GB and ark also says J4125 can just handle 8GB memory max. Is 16 or even 32GB possible?
@Michael Fuckner its 8GB in the VM, but there’s a screenshot of the machine that says it’s 16GB installed so the system is seeing 16GB in there.
@tiredreader My siblings and I were taught in school that using “one” instead of “you” is more formal so I’m assuming that’s why they’re using the style. It’s like the higher-education version of writing but I’ll agree most of the Internet is written for lower educated audiences. I don’t really care either way.
@tiredreader, I had to laugh because about a year ago I said same thing in the comments of another STH article. The overuse of “one” annoyed enough to comment on it. It’s not just Patrick but his entire writing staff use that “one” pronoun… alot.
Patrick probably gave a template doc to his writers and they follow it VERY closely. I’m sick of it too but I laughed when I read your noticed it too. It’s annoying as heck.
Dear Patrick – One shouldn’t be so annoying. ;)
Given that this is STH I do have to ask this question:
Are the Intel components in this device, especially the NICs, actually INTEL parts or simply cheep knockoffs created by some no-name Chinese manufacturer?
I agree with other commenters that say the rating system on STH is meaningless.
@Patrick just think what ads you could see by recovering the page space taken up by the meaningless ratings section!
Delete. Repurpose. Profit!
Four ports can be frustratingly just too few, especially with virtualisation… but 6-port versions are starting to appear, with a choice of either Core i3-8140U or Core i5-8260U. I’d be intrigued to see a comparison of the pfsense performance of one of these versus the celeron version, as comments for the celeron have suggested it is underpowered. (the 6-port version appears as “Industrial Soft Router Intel i5 8260U i3 8140U 6x 2.5GbE i225 LAN Fanless Mini PC AES-NI 4*USB Rs232 pfSense Firewall Appliance”, though the description doesn’t mention the stepping).
I do wish One (?) would get over this American obsession with self-gratification and dump the beast-tard pfSense. Their unethical (and judged “illegal”) behavior is legendary (and memorable through The Way Back Machine) with their dealings with the far better OPNsense (not .com). Their incompetence was stunning with their contribution of the Wireguard snafu. I am glad to see in this article that 84% of rated performance is judged “good” (or should I say 9+++?) with their i225 drivers. pfSense is now proprietary software, while OPNsense is truly Open with many contributions by the community. Their hardware is far superior. If only pfBlockerNG were ported, few would question the changed (except the America is Great camp).
It is odd that using pfSense should make me feel dirty.
In the article you talk about two m.2 slots, but in the AliExpress description of the appliance the SSD slot is marked as a mSATA and the WIFI slot as a mini PCI-e.
Can you confirm which one is true?
Can you provide a tutorial how to do the PVE MGT, PVE LAN (Pass thru means?), and FW LAN and FW WAN set up.
I am new to virtualization, and this subject is perplexing to a me. Thank you in advance.
Ha guys. Fun stuff.
1. That is correct, we use “one” because that has always been the proper writing style for a site like STH. STH did not grow from a gamer site and >90% of our reader base is over the age of 25 and has at least a college degree. I am not trying to engage the under 18-year-old audience as much with STH, and we have a sub 0.3% readership in that demographic (as well as YouTube.)
2. Fun fact, we do not use contractions on STH either.
3. No more ads Sleepy. Everything we have is there for a reason, and I specifically tell folks no more ads. See the recent Q1 2022 Letter from the Editor.
4. Jake – I think that was in our recent pass-through tutorial linked in this article.
5. On pfSense, we did a lot of OPNsense in this article. On the other hand, pfSense is still a much larger project and we have many readers that have used it successfully for years.
I’m satisfied regarding the over use of “one”. I feel better after getting it off my chest, knowing Patrick is aware that it irritates me so much. lol. I’m only speaking for maybe a fraction of my Cajun demograph of college grad 52+ yr old segment
The use of the pronoun is adequate Patrick and writers.
Your site, your rules, and nobody is forced to come and read. I’m clear on that.
However the criticism is on the overuse of it. It does dilute the air of professional writing you are trying to demonstrate.
I hope some minor criticism will be constructive.
While adherence to professional stylistic manuals is appreciated, pronoun usage in general is also discouraged in professional styles, including the “one” pronoun. This is especially true when on one hand, there is an attempt to have the writing style within a manual, then to add in personal style such as “fun fact,” which seems equally overused. I recognize every person has their own speaking or writing style, however in my opinion the disconnect only serves to distract.
In the early days of STH, reviews featured heavy homelab content, with less emphasis on rigid templates. This in my opinion made the the personal style work better. While many can appreciate the more enterprise content, the dearth of homelab content is a bit of a shame. At the end of the day, while IT professionals love new tech, it is largely unattainable as a hobby. Homelab content should also have enterprise/professional application/translation and vice versa, as this is the main reason many are into homelabbing.
The discounting of “gamer review” sites is a bit discouraging. Many so-called gamer review sites have had a professionalized bent in terms of reviews for over a decade now. We are not talking about joke click bait sites such as Wccftech, or even LTT, here. I find that some of those so-called gamer sites actually publish benchmark methodology and rationale. While gaming focused, the sites go through numerous tests. An example on STH where things can be done better is storage reviews are essentially a bunch of words along with a CrystalDiskMark screenshot. A viewer can just find plenty of CrystalDiskMark screenshots on Amazon reviews to obtain an idea. Another example are networking reviews that are a bit light on details, having spent a whole two pages on random facts that are not relevant at the end of the day.
Perhaps the market is moving towards YouTube/video content, but I think I can say for quite a few people that it’s annoying to need to watch a video to find out more information. Supplementary video content, or a full written review with accompanying full video review is preferable. Even a written transcript of the video would be fine. “18 year old” audience is a bit unnecessary to point out, so I will point out that video content appeals directly to these “kids.” I have not met a serious professional that prefers video content over written content.
I have been a loyal reader since STH’s early days. I will continue to be a loyal reader. I hope my light criticism isn’t taken out of context. The criticism is meant to be constructive, so keep up the good work.
I was today years old when I saw someone complain that there’s a lack of homelab content on STH when they’re commenting on homelab content.
I’d like to see just more content in general. I like STH’s homelab but they’re also the only ones really doing data center stuff for real right now.
I’ve just ordered what looks to be a slightly ‘newer’ version of this chassis/MB:
https://www.aliexpress.com/item/1005004011402622.html
It’s an N6005 Pentium Silver odel which claims to have 2 x DDR4 SODIMM slots (photos suggest there are definitely two) and an M.2 NVMe slot.
Options available go as far as 32Gb RAM and 1Tb NVMe drive, but I ordered mine barebones. I already have a spare NVMe drive so I’ll be using that but I will need to buy some RAM separately. What I’m confused about is what the maximum RAM that would be supported. My understanding is that the N6005 only supports up to 16Gb RAM officially, but there is a 32Gb RAM option available to purchase, and the advertising ‘images’ on AliExpress claim it can support up to 64Gb so I’m unsure whether 16Gb RAM is a soft limit or hard limit of the processor.
Is “up to 64Gb RAM” supported, or is it just another AliExpress ‘error’ that, when I buy more than 16Gb and it turns out to not work, won’t be their problem? Ideally the sweet spot for what I intend to use it for would probably be 32Gb but I can live with 16Gb if I have to, and I’d rather just know up front what it can actually support, rather than get 2 x 16Gb and then have to return it one of them when the second isn’t supported.
David – I was told the N6005’s are due out in May but the N5105’s motherboards are built so those systems are shipping.
Hmm thanks Patrick – that’s frustrating. No mention of that delay on their page or when ordering. I do now see on my order that it’s advising “This order will ship in 32 days. If not, your refund will be issued automatically.”, so it does seem you are right. Given I’m hoping to virtualise it and run a few other things, it might be worth waiting a bit longer.
Did you have any thoughts or answers with regard to the maximum supported RAM for the N6005 model? Is it 16Gb, or 32Gb+ or dependent on other factors that we can’t really know until we get our hands on one? ;)
Love it!
Really looking forward to a follow up with the Jasper Lake pentium, especially virtualised. I’m so temped to buy one but paranoid I’ll have issues with passing the NICs through individually so I eagerly await your attempts.
Keep up the great work!
David – Likely have to wait until they arrive.
I ordered one of the N6005 barebones (Topton) on the 16th as well. Not shipped yet. It is really frustrating when sellers do this surprise pre-order thing. The RAM and SSD I bought for this will be long out of their return window by the time this computer ships.
@David: I have bought the N5105 version and it works well with 2x16GB of RAM, running at 2933. I’m sure it will work just as well with N6005, I don’t know about 2x32GB though…
Hi guys i think i got fooled on ali express ??
i got a board called 1090NP-12 ver1.4
pdf of ver 1.0 (was not able to find 1.4)
http://m.e-bluetech.com.cn/?c=msg&id=2273
says that this board has i210 chip i am still not able to confirm
but from what i can tell this is a very old chip(2012)
https://ark.intel.com/content/www/us/en/ark/products/64400/intel-ethernet-controller-i210at.html
and lsiting will say i255
first thing that got me wondring is why is my board blue?
looking at the intel chips thay are diffrent number then showen here
still not sure if its better or not
can someone shade some light here ?
Have got a barebones N6005 on order from KingNovy PC Store, latest info I received was the’re in testing and they’re hoping to start shipments between 5th and 10th of May. So hopefully not to long now.
I have this board (v1.4 with i225 2x SODIMM) and have one minor issue in Proxmox.
NICs are listed as:
2: enp1s0:
3: enp2s0:
4: eno1: altname enp3s0
5: enp4s0:
All works OK but just a bit annoying.
Anyone know of a BIOS update or whether Coreboot/Seabios work?
thanks anthony i was afarid i got fooled
anthony can you tell me if you where able to get the 2.5 speeds?
Any of you that have the N6005 model and tried to use 32GB+, I would be curious to know if your unit can successfully complete a full pass of memtest86+. I have the N6005 Topton model and it will recognize and boot with 2x16GB modules, but it will not complete a pass of memtest86+. The HDMI output eventually becomes scrambled and the unit reboots. If I test either 16GB module individually in the unit or use 2x8GB modules of the same make/specs, the unit will successfully complete 4 passes of memtest86+. Topton advertises that it will support 64GB but that is contrary to ark.intel.com which shows a maximum of 16GB supported on the N6005. Given that every run of memtest86+ results in the same graphical corruption and then a crash, I suspect that it has something to do with iGPU memory addressing, but I would be interested in hearing if anyone else has the issue with 32GB or greater installed.
@Gad – Sorry but it is the only equipment I have that is faster than 1gbit.
Thanks @Anthony
do you have firewall installed there ? it shuld tell you the basespeed (dashboard + Intrecfaces )
1000baseT
i think it will be good to check so we know we are not getting ripped off…
also for others … i did not get my RAM stick yet machine is down :( for me
Very foolish using this as a firewall when it doesn’t even come with a console port (preferably RJ-45 cisco/rollover pinout), and I doubt it has any CLI-based BIOS like coreboot.
I’ll wait until Protectli make a proper firewall appliance out of 2.5Gbe NICs rather than shooting myself in the foot with one of these.
Recommend you forgo “quick pre-holiday weekend pieces” in future, and just do proper in-depth reviews. These shallow reviews aren’t worth anyone’s time.
@Gad I’ll update once I have it installed and working. Proxmox is detecting 4x i225 though.
@Gavin Owen – There is an onboard serial port ;) If using a VM firewall though it is not so important. Protectli while nice, it should be for 4x the price and not really ‘home’ oriented.
cool thanks again… the all reason i bought this system if becuse of the i225
@Anthony i just got my ram stick …
and when trying to boot i dont get anything … when connect a keyboard all lights are on
like MB is stuck
removing the memory will result in beeps … so i guess memory is ok… is there any jumper maybe to switch anything on?
how much ram do you have? i suspect my RAM stick is bigger then the board can handle
I bought this one, specifically because it advertised as having 2 RAM slots: https://www.aliexpress.com/item/1005004130282808.html
No idea what I’m actually going to get; I didn’t even realize there was a version with an N5105.
I think this will be plenty for my needs; it’s the same processor as in a QNAP TS-253D NAS, and twice the cores of my RT2600ac router, which is a dual core device. (My NAS, btw, has no problem recognizing and using 16 GB of RAM (2×8), even though its officially limited to 8 GB.)
Does this thing take 2280-size SSDs?
@Gad – I have tried 2x 4Gb DDR4 2400 and 2x 8Gb DDR4 2666, both are non ECC and work fine.
Memory Device
Array Handle: 0x0023
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 8 GB
Form Factor: SODIMM
Set: None
Locator: A1_DIMM0
Bank Locator: A1_BANK0
Type: DDR4
Type Detail: Synchronous
Speed: 2400 MT/s
Manufacturer: SK_Hynix
Serial Number: 32370923
Asset Tag: 9876543210
Part Number: HMA81GS6JJR8N-VK
Rank: Unknown
Configured Memory Speed: 2400 MT/s
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V
Have you tried clearing the CMOS?
I read that someone tried 32Gb which booted OK but the MEMTest always failed. 16Gb is the officially max supported I believe.
It also requires a few secs boot delay to detect USB devices ;)
@John – This uses either MSATA or SATA SSDs not M2/NVME. I’m using Transcend MSA230S 64 GB
@Anthony
Hi Thanks
i think i will send it back … seller tells me to get another brand of RAM (Strange for me)
i think its related to that i bought 8gigs and not 4*2 for this board… i guess i could not know :(
anywhy i am not going to buy another ram and waste time and the dispute will go to waste …
and i have no other SODIM DDR4 to check this so i am out of options
i am a litle disappointed from the seller KingNovyPC Computer Store, he was acting like i dont like his pc
and igonred that i am stuck with ram and one month of a waste of time
also he had the idea that i will pay for the return shiping … but aliexpress stepped in
All of these J4125 devices I have seen have the following in their description so it was mentioned.
Barebone bundle had best to use Samsung, Sky Hynix etc ORIGINAL brand ram and storage.
I have 2x 8Gb RAM working fine. It is 2666mhz but running at 2400Mhz so it can work fine with good quality RAM. I have been running it for a few weeks now and it has been 100% stable so far.
@Mikes Did you try memtest86 (one of the closed source/freemium UEFI versions)? If the board firmware has messed up the 32 bit memory address table, then memtest86+ might end up overwriting graphics RAM. Maybe the UEFI memory map is OK? I’ve had memtest86+ stray outside physical RAM regions (into memory-mapped I/O regions) in the past and give bogus failures, when other RAM testers worked OK.
@Tim So, oddly, the system runs fine with 2x8GB 2666MHz modules (Mushkin). The system does not like 2x16GB 2666MHz modules. I tried 2 different 32GB (2x16GB) 2666 kits, one from Mushkin with the same specs as the 16GB kit, and one from Team Group. Both resulted in unexpected crashes. I tried a third 2x16GB kit, this time 2400MHz from Samsung, and it works fine. Full passes of memtest86 succeed and guests run fine on Proxmox. I don’t remember the version of memtest I used but it passes 4 runs when the Samsung memory is installed.
@Mikes – Please provide to part# for the Samsung working RAM.
so one last note…. I bought from Kingdean what ever their name is … i did not give the store a bad rep… but thay deverse one… thay asked me to return the prodect on my shiiping fee.. and thay will rerutn the money when thay get it back, and what do you know … thay are not answering me anymore … just dont buy from there … thay have no idea what thay are giving you …
Thread resurrection :)
I have had a 1gbit fibre service installed and the box connects to the GPON 10gbit port at 2.5gbit without issue.
The Proxmox OPNSense has no issues routing at 1gbit but with NTOP-NG enabled at that speed it does use a lot more CPU.
Picking one up later today – you guys moan alot!
A man was confused with SO-DIMM slots because notebooks use same height dual SO-DIMMs connector.