MikroTik is pushing into the 100GbE and 25GbE market, and has something that is quite exciting: a bigger option. The MikroTik CRS520-4XS-16XQ-RM is a 16-port 100GbE and 4-port 25GbE switch with a surprisingly beefy management processor. The best part, it has reasonable performance and power consumption and a new price tag at $2195, likely meaning we will see a sub $2000 street price. This is a big launch for the company, so let us get to it.
MikroTik CRS520-4XS-16XQ-RM Overview
For this one we have a video that we always suggest watching in its own tab, window, or app for the best viewing experience:
A few funny things happened on the road to this review. First, the switch was sent by MikroTik and got stuck in US customs for a surprisingly long time. Second, Patrick was at the AMD Zen 5 RDNA 3.5 and XDNA 2 event in Los Angeles. While in LA, someone recognized him as “the guy who reviews MikroTik switches” (so clearly a YouTube viewer.) He was asked about this exact switch by the gentleman because he runs a studio and needs to connect maybe a dozen editors to storage, and something like this would be the perfect device to do so.
A big challenge in a device like this is that it is only about 3% of the bandwidth of a large switch today, so there are other 32-port 100GbE switches on the market used for under $1000. MikroTik needs to differentiate between its new $2195 offering, and the cheaper and higher capacity used options. We are going to go into how that was accomplished.
MikroTik CRS520-4XS-16XQ-RM External Hardware Overview
The 1U switch is going to be great for some, and challenging for others. I think many will want a half-width switch so 1U can handle redundant switch options. On this switch, it would have meant sacrificing data ports and features to make that happen, so we understand why it is a full 1U design.
The big feature, of course, is the set of sixteen 100GbE QSFP28 ports. In the management interface, these show as four bundled 25GbE lanes, so things like breakout cables and QSFP28 to SFP28 adapters can be used.
If you simply want 25GbE, there are four SFP28 ports.
Next to that we get a console port and reset button. There are then two 10Gbase-T ports. These 10Gbase-T ports go directly into the Annapurna Labs management processor not to the Marvell switch chip, so there is an additional hop in the chain to them.
On the rear of the switch, we get fans and power supplies.
Mikrotik has little hot-swap fan modules, MT-HotSwapFan. There are four of the modules in this switch.
The power supplies are MikroTik PW-12V150W units. Each is rated for 150W.
These are MikroTik branded hot-swap power supplies, not third-party units. Moving to a half-width switch would have impacted both the front port count and likely also the ability to have redundant hot-swap power supplies in this switch.
As a quick note, we show the rack ears in the video. We did not take photos with them, but they are in the box.
Next, let us get inside the switch to see what is going on.
If you want to use a full 100gbe you kinda need RDMA for windows. I have a 32 port 100gbe and most windows transfers max out around 40gb. Which is fine, I’m sure this switch works great as a 40g switch as well. I also hacked 120mm noctua high pressure fans into the top lid making it very much a TOR router by necessity. So something like this would be great.
However I didn’t go with iwarp compatible nics so there is a decent bit of switch compatibility required. Will they support PFC on the feature roadmap?
I wonder why so many switches go for the 1U full width form factor. You’d think if you went with 2U half width then you could get the same port density by putting two side by side, but you’d be able to use larger, more efficient fans and heatsinks, and quieter fans as well.
Hi, did you test the switch with high power transceivers?
Does it actually support modules with higher power classes. E.g. 80km 100G ZR4 modules will use 5.5W. Some devices will refuse to operate transceivers with a higher consumption than 3.5W.
Given the low power budget left for transceivers it seems like it’s designed for use with only twin ax DAC cables. Even 16 x low power 2.5 watt SR4 transceivers would be a problem. Is there any official word from mikrotik regarding transceiver power budget?
RouterOS 7.15 seemed to have added some initial support for PFC, and 7.16 seems to be refining a bit more.
Here’s the thread on QoS:
https://forum.mikrotik.com/viewtopic.php?p=1089316#p1089316
You can see I asked about clarification on RoCE towards the bottom but haven’t received an authoritative response – only a community member stating “You don’t need DCB support, just use RoCE v2” which I don’t consider to be a great answer. Maybe if yall want to ask as well and help me bring focus?
I’m sure Patrick / STH is also asking Mikrotik, as it came up in previous videos with the 4 port 100Gb CRS 5xx switch.