Supermicro AOC-S25G-b2S 25GbE NIC Review

The Supermicro AOC-S25G-b2S is a dual-port 25GbE Ethernet controller designed for servers. You may have seen that we recently tested the AOC-S25G-i2S. That “i” changing to a “B” in this model signifies that instead of using an Intel chipset, this NIC uses a Broadcom chipset. Specifically, Broadcom BCM57414 is used. Given today’s trends toward using 25GbE, we wanted to take a look at another option.  In our mini-review, we are going to take a look at the card and what it has to offer.

Supermicro AOC-S25G-b2S Overview

When we first view the Supermicro AOC-S25G-b2S one can see a PCIe Gen3 x8, low profile card. We have two SFP28 ports for dual-port 25GbE connectivity. This mirrors the PCIe Gen3 x8 slot’s bandwidth more closely than a quad 10GbE card or 40GbE card because one has enough PCIe bandwidth, without oversaturating the bus.

Something that will be immediately noticeable is how much smaller the heatsink is compared to what we have on the Intel alternative AOC. It is roughly half the length on the Broadcom card versus the Intel card.

That is because this is a 2W lower typical power consumption card versus its Intel counterpart (5.2W v. 7.2W.) It may only be two watts, but that is a lot on a percentage basis which also means a smaller heatsink. For some context as well, the Mellanox ConnectX-4 Lx dual 25GbE solution has an 8.7W typical power. Broadcom is the low power leader here.

For those wondering, that connector on the top right of the card is the NC-SI header that allows this card to also handle out-of-band network duties for a server’s IPMI / BMC.

While on the Intel card we had a debugging mode connector, on the Broadcom 57414 we get a 3.3V standby power jumper that turns that feature on or off. For most of our readers, this is not a jumper that will be changed, but it is likely more useful than the debugging jumper on most NICs they deploy.

We have not yet looked at a Broadcom 57414 NIC, but we did previously review the Dell EMC 4GMN7 Broadcom 57404 25GbE NIC. This Supermicro NIC is a newer generation that adds a number of new features that we are going to get to a bit later in this piece. Something that the Broadcom 57414 offers that the Intel XXV710 and BCM 57404 did not offer is RoCE offload support. Mellanox has since its ConnectX-4 Lx generation incidentally. Still, it is a feature more common in second and third generation 25GbE adapters, like the BCM 57414 on this Supermicro AOC than first-generation models.

The rear of the card has only a handful of small surface mount components.

On the dual-port NIC, we find two SFP28 ports. These look similar to traditional SFP+ ports but offer more bandwidth with the ability to handle 25GbE on each port. Just as a QSFP+ port could split into four SFP+ ports, a QSFP28 port can split into four of these SFP28 ports.

One will also notice that this card utilizes passive cooling. Even with the smaller heatsinks it is not a completely passive solution insofar as the solution is designed to have airflow over the heatsink. If you wanted to, for example, use these in a low-power embedded node that Supermicro makes, that chassis will need to provide airflow to the heatsink side of the NIC.

Supermicro AOC-S25G-b2S Drivers

The Supermicro AOC-S25G-b2S utilizes the Broadcom bnxt_en driver. That means in newer distributions of Linux and other OSes it will be automatically recognized and work out of the box. Since this NetXtreme-E controller is fairly popular, most modern OSes have built-in support for this NIC. Some older OSes will not. It is always good to double-check when specifying NICs in servers.

Although we have discussed the card as being 25GbE, it also supports both 10GbE and even 1GbE. Getting the dual-port model and stepping down to 1GbE for WAN connectivity can make sense versus getting two single-port NICs.

We tested the 10GbE compatibility with a number of as well as six different SFP+ to 10Gbase-T adapter series converters. The six we tried worked fine but they use a lot of power relative to the card. If you have a 10GbE (SFP+ or 10Gbase-T) network today but are upgrading to 25GbE in the server’s lifetime, this can be an easy way to ensure there is an upgrade path available.

We are going to punt a bit on features to the Broadcom documentation. The feature lists of modern NICs are so long that we suggest searching for what you want. One nice feature with the BCM 57414 is that Broadcom aggressively marketed the solution against the Mellanox ConnectX-4 Lx. Meanwhile, Mellanox marketed its ConnectX-4 Lx heavily against the BCM 57414. That means there is plenty of competitive analysis highlighting differences. Supermicro has both options in this form factor along with the Intel XXV710 solution.

Above you can see key specs for the card.

Supermicro AOC-S25G-b2S Performance

We tested the Supermicro AOC-S25G-b2S using a 1m breakout DAC to our 32-port 100GbE Dell Z9100-ON data center switch. We then varied the port speed and connection type for iperf3.

Overall, these are great results. One can see a performance where we would expect a newer generation adapter to stand. The 25GbE link is faster than dual 10GbE links which make sense. We are seeing lower power to reach this level of performance than we did on the Intel card as well.

Final Words

Overall, the AOC-S25G-b2S performed well in our testing. It is a true competitor to the popular Mellanox ConnectX-4 Lx which Supermicro offers in its AOC-S25G-m2S as well. With RoCE support as well as many offloads, it is certainly a more modern NIC than the Intel XXV710 solution which is readily apparent from the feature sheets.

This year, we are fully behind transitioning to 25GbE at STH. If you are making a leap from 10GbE, it may be wise to get a NIC like the Broadcom NetXtreme-E based Supermicro AOC-S25G-b2S that supports newer features with offloads as well. Where the AOC-S25G-b2S differentiates itself from other BCM 57414 offerings is that it is designed with features such as the NC-SI header as well as the monitoring and management features for Supermicro servers.