The Supermicro AOC-S25G-i2S is a dual-port 25GbE Ethernet controller designed for servers. Given today’s trends toward using 25GbE, configuring your server with this card can make a lot of sense if your deployment is utilizing the newer standard. In our mini-review, we are going to take a look at the card and what it has to offer.
Supermicro AOC-S25G-i2S Overview
Looking at the Supermicro AOC-S25G-i2S one can see a PCIe Gen3 x8, low profile card. With only two SFP28 ports the card is able to maintain a low profile form factor allowing it to be deployed in full height or low profile slots. The PCIe Gen3 x8 bandwidth relatively closely mirrors the 50Gbps of total port speed on this device. That was a considerable drawback with the dual 40GbE PCIe Gen3 x8 cards insofar as the network bandwidth on the Fortville controller could outpace the card’s PCIe bandwidth.
For those wondering, that connector on the bottom right of the card is the NC-SI header that allows this card to also handle out-of-band network duties for a server’s IPMI / BMC.
We first saw the Intel XL710 series it was a 40GbE controller. At the time, STH noted Intel Fortville had Lower Power Consumption and Heat output compared to its peers. As the market shifted from 10GbE and 40GbE towards 25/50/100GbE, Intel needed to pivot. The company has been showing its Intel Ethernet 800 Series 100GbE NIC for about a year, but it is designed for PCIe Gen4 servers which Intel does not sell as of this writing.
The rear of the card has only a few small surface mount components. This is important if one is guiding airflow to hotter expansion cards such as GPUs.
On the dual-port NIC, we find two SFP28 ports. These look similar to traditional SFP+ ports but offer more bandwidth with the ability to handle 25GbE on each port. Just as a QSFP+ port could split into four SFP+ ports, a QSFP28 port can split into four of these SFP28 ports.
One will also notice that this card utilizes passive cooling. Although we found Fortville to be a lower power Ethernet controller, these cards rely upon chassis airflow to keep them at proper operating temperatures.
Supermicro AOC-S25G-i2S Drivers
The Supermicro AOC-S25G-i2S utilizes the Intel i40e driver. That means in newer distributions of Linux and other OSes it will be automatically recognized and work out of the box. The XL710 series (40GbE) utilizes this driver and that controller was a 2014 part. As a result, most OSes have built-in support for this NIC. Of course, you can install an updated driver, but if this is your only networking device, it is good to know that you can at least reach the network to fetch an updated driver.
Although we have discussed the card as being 25GbE, it also supports both 10GbE and even 1GbE. Getting the dual-port model and stepping down to 1GbE for WAN connectivity can make sense versus getting two single-port NICs.
We tested the 10GbE compatibility with a number of optics (we tested this card before our recent SFP+ to 10Gbase-T adapter series started) and they worked fine. If you have a 10GbE network today but are upgrading to 25GbE in the server’s lifetime, this can be an easy way to ensure there is an upgrade path available.
There are a few features that are conspicuously absent. For example, with these NICs one can get up to 256 VMDq VMs, 128 virtual ports with SR-IOV, 4096 VLANs, and both VXLAN and NVGRE. One does not, however, get RDMA and iWARP. Although this is a 25GbE adapter, it was largely built with Intel’s 2014 feature set. If you were wondering the “i” in -i2s” means Intel.
Supermicro AOC-S25G-i2S Performance
We tested the Supermicro AOC-S25G-i2S using a 1m breakout DAC to our 32-port 100GbE Dell Z9100-ON data center switch. We then varied the port speed and connection type for iperf3.
Overall, these are great results. One can see that the single 25GbE link is faster than dual 10GbE links which makes sense. Even though this card uses the smaller SFP28 connectors, it is able to reach performance nearing dual 40GbE adapters even in a 7-10W power impact to the server it is installed in. That helps increase network density in racks.
Final Words
Based on a mature Intel XXV710 network controller, the Supermicro AOC-S25G-i2S is a solid option for those looking to design their next-generation servers with 25GbE connectivity. This year we expect a strong ramp in SFP28 25GbE infrastructure which is going to make doing so more practical for many organizations. The Supermicro AOC-S25G-i2S is a great step in that direction if the feature set of the card supports your organization’s needs.
Is there any way we can see some juxtaposition with the Chelsio T6225-CR and Melonox connectX-4? I’d be interested to see how they stack up.
Hi Joe. We do not have a Chelsio T6225-CR at this point and I think the ConnectX-4 Lx’s are all in servers. Great idea on doing a comparison piece. Let us see if we can get materials together for it.
Transcievers support is also a thing. Some Intel cards are picky depending of the driver. If you test mellanox please include the xconnect5 as well (besides 4).
25Gbit/s is is 3125 MByte/s. Your test shows this thing managing just over 2000 MByte/s for a single link.
Where do you suppose the other 1125 MB/s are lost? Is it limited by the host server PC or the switch?
Hoohoo if you’re just dividing bits to bytes you’ve also got to build in Ethernet framing and overhead to get to how much data is transmitted.
The Intel XXV710 has been causing me one issue for over a month, with a Rome-based demo 2U/4node SuperMicro quad…And pending server deployment: When a server sled is inserted into a chassis, and the server gets far enough into the POST/boot-up process to try to PXE boot off the network, the card will say it doesn’t see the cable…ctl-alt-del, and instantly the NIC links lights come on, and once we are back to the PXE boot off the NIC stage, the NIC sees the cable and I am off to the races…Different switches, different cable types/brands (100/25 breakout, 25 gbit twinax to a new 25G switch). The issue still occurs…SM swore that no one else had this issue…Once I tested a 10g connection to this NIC, and that didn’t have the power-up issue, SM admitted it saw this issue on a couple of models using this Intel NIC…Still not sure IF Intel is taking this seriously…Since I will be running a bunch of these servers with a single network cable, after a power issue, I would lose remote access to said servers. That’s a problem.
21:00.0 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
21:00.1 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
Very nice. But boi these bad boys are expensive. I wonder what the plan is for the future of these? 25Gb, then to 50GB? OR straight to 100Gb? All in all, nice article.
@nelson.w, thanks! But the Ethernet header + CRC + interpacket gap is max 42 bytes. Which does not account for the difference (assuming a full sized payload).
On the photos I see the card has a Nuvoton management chip in the upper right corenr. How is this useful? I have an X540 card (SUPERMICRO AOC-STG-I2T) which has such chip but I cannot find any way to use it. In particular, lm-sensors doesn’t find it (although I don’t know if it should). Anyone has any more info on that?
I really want 25GbE but have yet to see any switches that would be cheap and quiet enough for the homelab.
Maybe in another 6-10 years. Or make your own low port count switch out of a bunch of these cards and vyos\pfsense or similar. #lightbulb
I did a little more testing testing this week…This time with a Mellanox 25gbit switch…Not perfect, but much better results.
If the Mellanox switch is set to 25g-only, the NIC behaves the same as with the whitebox switch in my above comment, however, if the switch is set to auto-negotiate, then the NIC will power up at 10g…The server boots up fine, sees all its NFS mounts…Only issue: It is running at 10g vs 25gbit.
If at this point I reboot, the NIC comes up at 25g…This is a much better situation of course, given that I have full remote access (ssh, IPMI) post power-up and can easily resolve the speed issue…Versus in the scenario of my 1st comment of having to recover from a power-outage or server insertion into a 2U chassis, via a crash-cart.