It was a cold day in February when we were in London to cover the AMD EPYC Embedded 3000 Series Launch. On paper, the platform sounds positively promising. We saw the AMD EPYC 3000 embedded part after a lot of sleuthing at Computex as we showed in the article: Piecing Together the iEi Puzzle AMD EPYC 3000 Spotted in the Wild. Today we have an exclusive and exciting bit to share: hands-on with the AMD EPYC 3000. Specifically, we have the AMD EPYC 3251 8-core single die part and have generated a few numbers.
Wallaby with the AMD EPYC 3251
For our review, we are using the AMD EPYC 3251 on the AMD Wallaby platform. Generally, we do not use development platforms for reviews, but this is the first platform we have been able to get. It has taken months of pestering anyone we could. Here is the AMD Wallaby platform in all its glory with the single die AMD EPYC 3000 CPU. One can see that there are many features not present such as an additional PCIe slot and additional RAM slots. We think that these would be placed with a dual-die configuration.
There are a few features we wanted to point out. First, as a single-die AMD EPYC 3000 series platform, one has access to two DDR4 DIMM channels. One also gets 32x PCIe 3.0 or high speed I/O lanes. There is no need to parse these lanes out for Infinity Fabric like on the larger AMD EPYC 7000 series CPU. We have a total of 24x PCIe lanes exposed via slots with the Wallaby platform.
As a reference, here is the AMD EPYC 3000 series SKU list from the launch event. You can see that the AMD EPYC 3251 is the 8 core, 16 thread part in the middle of the SKU stack.
This is the top-end single die AMD EPYC solution at the moment, so we wanted to explore what it has to offer.
AMD EPYC 3000 Series Networking
Here is a fun fact, each AMD Zen die has up to 4x 10GbE MACs onboard. On the desktop parts, these are not used. On the AMD EPYC 7000 series, these are likewise not used. As Cavium learned with the ThunderX (1) generation, and AMD learned with the EPYC 7000 series, cloud providers tend to like using their own NICs. Even companies like HPE and Dell EMC utilize LOMs instead of the Intel C620 series chipset NICs.
The AMD EPYC 3000 series is interesting because it targets the embedded industry. The embedded industry is a different market segment because appliance vendors tend to make the entire stack from appliance to software. Here, the embedded 10GbE NIC have a chance to be used. The AMD Wallaby platform is the first that we have seen in over a year of EPYC and over a year and a half of Zen that we have seen the AMD 10GbE NIC.
Now presenting, the AMD Zen 10GbE NIC:
Our AMD Wallaby platform has dual SFP+ cage NICs. We were able to pass 10Gbps of traffic through the NIC, but we wanted to focus more on the CPU performance and looking at more of the platform.
AMD EPYC 3251 8 Core SoC
We are going to get to performance later in the article. We wanted to first show that this platform supports virtualizations and feature sets that align with the AMD EPYC.
There are a total of 8 cores. With 2-way SMT that gives us 16 threads. AMD’s SMT, unlike Intel’s Hyper-Threading, is not susceptible to the L1TF Foreshadow security vulnerability. That means if you are running untrusted code VMs on your embedded platforms, you may have to turn off Hyper-Threading on Intel to make the Xeon D-1500 or Xeon D-2100 more secure. With the AMD EPYC 3000 series, you do not have to do so.
Base clock speeds are 2.5Ghz for the AMD EPYC 3251 part with maximum all core turbo boosts of 3.1GHz. As we will see, the increased clock speeds help the solution perform very well. TDP on the AMD EPYC 3251 is 55W (note current spec is 55W, originally was 50W) putting it between the Xeon D-1500 and D-2100 series. L3 cache comes in at 16MB which is much larger than the Xeon D-1500 and comparable to the combined L2/ L3 cache setup of the Skylake-SP derived Xeon D-2100 cores. The net result is a relatively power efficient part with the capacity to deliver a lot of performance.
Do ya’ll have the 16 core one?
I’m going to ask our PM team if their fixin’ to make up our appliance with embedded 3000 EPYC.
I’ll read the rest later, but you’re right on the need for an Intel alternative. These side channel attacks and the embedded clock bugs show a need in the market for a second x86 supplier.
Can someone make a damn mini ITX motherboard with these, 4 DIMM slits and some NVMe headers?
Are the single die parts way cheaper? The higher core counts looked like huge cost savings but I’m not seeing any on AMD’s site. Only 4 SKUs there https://www.amd.com/en/products/specifications/embedded/8161
Model: AMD EPYC™ Embedded 3251
Product Type: SOC
Family: AMD EPYC™ Embedded Processors
Line: EPYC Embedded SOC
OPN: PE3251BGR88AF
TDP: 55W
CPU Type: Zen
CPU Base Freq.: 2.5GHz
CPU Max Freq.: 3.1GHz
# of CPU Cores: 8
# of Threads: 16
GPU Support: No
Security Processor: Yes
Total L2 Cache: 4MB
Total L3 Cache: 16MB
System Memory Type: DDR4
DDR4 Rate (Max): 2666 MHz
Memory Controller: Dual Channel
ECC: Yes
USB 2.0: 0
USB 3.0: 4
USB 3.1 Gen1: 0
USB 3.1 Gen2: 0
SATA: 8
Low Speed Interfaces: EMMC, eSPI, GPIO, I2C, LPC, SMBus, SPI, UART
# of PCI controllers: 8
Gen3: 32
Gen2/3: 0
Gen2: 0
Ambient Temp Range: 0-105°C
Enhanced Temp Support: Extended Temp (0-105°C)
Infrastructure: SP4
Last Time Buy: 2028
Recommended for new designs: Yes
Unless I missed something, but who makes the NIC? The lspci output showed the NIC as an AMD network card, but who did AMD license the NIC from? What network driver is it the chip using? That could provide some clues.
RC it’s on the first page showing as an AMD network device so that’s AMD NIC IP.
I’m just surprised AMD building their own NIC IP. I suspect they licensed it as many others do. E.g. Asmedia makes the chipsets.
Great review.
RC it’s a SoC.
Hi Misha, I understand it is an SoC. Typically what you do as part of SoC designs is license IP (in chip design IP means a blob of VHDL / Verilog code), so let’s say I design an ARM SoC, I get an ARM core from ARM, I may get a video core IP from Imigation, a flash controller IP from someone else. Yes, you can design pieces in-house too. Something like networking is so complicated and I don’t recall AMD having any experience in there, I doubt they started from scratch and just licensed IP from a Marvell or some other company. I’m curious what IP it is unless it is truly their own design.
I found the answer to the AMD NIC question. I noticed the screenshot showed ‘amd-xgbe’ driver, so I looked at the Linux kernel source. Based on the code AMD licensed “Synopsys DWC ETHER XGMAC”, see https://www.synopsys.com/dw/ipdir.php?ds=dwc_ether_xgmac. I don’t know how much this IP does as it provides a MAC layer, so AMD likely had to build some parts themselves, but I’m not sure.
Finally, Patrick, thanks.
I would assume that’s $315 USD in bulk lots of 1000 for the chip? Wholesaler price?
So could we assume at least $550 / $600 US motherboards using this?
I’d still strongly consider it, it certainly seems to perform. I’d prefer perhaps something 15w lighter and $100 lighter, but we’ll have to wait for more boards.
As I posted on the forums, incredibly hugely disappointing that AsRock, SuperMicro, Tyan all either told me “nope, no interest in this!” or flat out didn’t respond!
These would make a monster FreeNAS machine, especially the 3201 model with 8c/8t at only 30Watts!
Patrick, do you know if this problem impacts the Epyc 7xxx or 3xxx series?
https://bugzilla.kernel.org/show_bug.cgi?id=196683
https://forums.freenas.org/index.php?threads/ryzen-stability-on-11-0-u4.59017/
$315 is the tray pricing. Anyone buying these embedded parts is likely buying quantities for motherboard runs.
I believe that kernel bug is for Ryzen users and has not been confirmed on EPYC.
what do you think, is Intel any part of delaying the go-to-market for these chips?
@kpin:
I hadn’t thought of that, it seems unlikely but I wouldn’t put it past them to try that.
I did contact SuperMicro, Tyan and AsRock about this CPU multiple times since Feb and have either had no response or an outright “no” from them, which was disapointing.
However, SuperMicro already do some AMD Epyc stuff, so they aren’t ‘frightened’ of Intel, to my knowledge.
We need a competitive board to the likes of SMCI’s x11sdv-*c-tlnf8 boards. (I hope I got the model name right from memory.)
Would be great if we could get the same treats the Intel line currently has.
Anyone know of a board coming out with the dual 10 Gb option? the supermicro boards which just came out are all 1 Gb.
@Nathan
Asrock Rack EPYC3251D4I-2T supports dual 10 GbE NIC speed (and M.2 slot supports 22110 which is unique to my understanding).
See https://www.asrockrack.com/general/productdetail.asp?Model=EPYC3251D4I-2T#Specifications
cpu socket ???
norbi31 – no CPU socket. These are embedded parts.
Any sign of the higher level SKU 3451, 3401, 3351,3301? that is not COM Express Type 7? And they seem vaporware as well.
https://www.amd.com/system/files/documents/3000-family-product-brief.pdf
Steve! Yes. Stay tuned in a few weeks to STH.
Any updates. This platform looks very cool, and I would like to use it for routers and small appliances, but I can’t find any consumer motherboard or system that actually exposes the built in 10Gbps NICs as SFP+. I did found some boards from Supermicro and ASRock Rack, but they all use Intel NICs for their 10Gbps and 1Gbps ports. Weird. Would be great to see native 4x SFP+ and 2x SFP (via external controller), and this SoC can do it quite well and cost competitively, yet I can’t find products doing that. Weird.
Some information about idle power use to someone like me who uses the platform in residential setting where the system is running 24/7, workload is bursty and idle power use is very important.
With the following configuration I’m getting 22.3W at idle.
M11SDV-8C-LN4F with EPYC 3251.
Stock fan replaced with Revoltec RL036 (mostly due to noise reduction, but likely it helps power use too).
64GB DDR4-2400 RAM (2x32GB HMA84GR7MFR4N-UH)
WD SN750 1TB NVME SSD
Only single gigabit network cable is connected. BMC is left unconnected.
PSU is 120W 12V power brick (Chieftec CDP-120ITX).
VGA is disabled via jumper (saves 1W)
“powertop –auto-tune” has been run (saves 0.2W if VGA is enabled, 1.1W if VGA is disabled)
The measurement has been made on console login screen of stock Debian 11 that had powertop installed. The power meter is not calibrated.