AMD EPYC 3251 Memory
The first time that we tried an AMD Wallaby platform, we saw the AMD EPYC 3251 performed slightly lower than what we expected. The reason for this was perhaps memory bandwidth. We found the platform running at DDR4-2400 speeds.
A quick conference call later, and the memory was running at DDR4-2666/ DDR4-2667. That is a big deal.
The AMD EPYC 3000 series supports two memory channels per die. The higher-end AMD EPYC 3000 SKUs support quad channel memory across two NUMA nodes. That quad channel memory is analogous to the Intel Xeon D-2100 series quad channel operation. The single die parts, like the AMD EPYC 3251 we are testing has two memory channels. If you compare this to the Intel Xeon D-1500 line, it is similar, but with an important difference. AMD EPYC 3251 8-core parts can utilize DDR4-2667 while the Intel Xeon D-1541 8-core parts can only utilize DDR4-2400. Also, we showed above that if you are designing an AMD EPYC 3000 series appliance you can use RDIMMs as well as use different speed DIMMs, perhaps if you do not need the bandwidth and want to save a slight amount of power.
AMD EPYC 3251 Virtualization and Containers
As one may expect the AMD EPYC 3251 can run virtual machines and containers. Since this is the first review online of the platform, we wanted to at least show that it works. Here is CentOS 7 (1804 iso installer) running in a virtual machine on an Ubuntu KVM host:
Since the trend is toward containers, here is the system running Redis in a Docker container:
One of our readers asked about ACSCctl for IOMMU which is important for virtualizing hardware these days.
As you can see, the platform has ACSCtl support for IOMMU. This is all expected since the AMD EPYC 3251 is built upon Zen which underpins the AMD EPYC 7000 series.
Next, we are going to move onto our benchmarking, power consumption, then discuss the competitive landscape before giving our final thoughts.
Do ya’ll have the 16 core one?
I’m going to ask our PM team if their fixin’ to make up our appliance with embedded 3000 EPYC.
I’ll read the rest later, but you’re right on the need for an Intel alternative. These side channel attacks and the embedded clock bugs show a need in the market for a second x86 supplier.
Can someone make a damn mini ITX motherboard with these, 4 DIMM slits and some NVMe headers?
Are the single die parts way cheaper? The higher core counts looked like huge cost savings but I’m not seeing any on AMD’s site. Only 4 SKUs there https://www.amd.com/en/products/specifications/embedded/8161
Model: AMD EPYC™ Embedded 3251
Product Type: SOC
Family: AMD EPYC™ Embedded Processors
Line: EPYC Embedded SOC
OPN: PE3251BGR88AF
TDP: 55W
CPU Type: Zen
CPU Base Freq.: 2.5GHz
CPU Max Freq.: 3.1GHz
# of CPU Cores: 8
# of Threads: 16
GPU Support: No
Security Processor: Yes
Total L2 Cache: 4MB
Total L3 Cache: 16MB
System Memory Type: DDR4
DDR4 Rate (Max): 2666 MHz
Memory Controller: Dual Channel
ECC: Yes
USB 2.0: 0
USB 3.0: 4
USB 3.1 Gen1: 0
USB 3.1 Gen2: 0
SATA: 8
Low Speed Interfaces: EMMC, eSPI, GPIO, I2C, LPC, SMBus, SPI, UART
# of PCI controllers: 8
Gen3: 32
Gen2/3: 0
Gen2: 0
Ambient Temp Range: 0-105°C
Enhanced Temp Support: Extended Temp (0-105°C)
Infrastructure: SP4
Last Time Buy: 2028
Recommended for new designs: Yes
Unless I missed something, but who makes the NIC? The lspci output showed the NIC as an AMD network card, but who did AMD license the NIC from? What network driver is it the chip using? That could provide some clues.
RC it’s on the first page showing as an AMD network device so that’s AMD NIC IP.
I’m just surprised AMD building their own NIC IP. I suspect they licensed it as many others do. E.g. Asmedia makes the chipsets.
Great review.
RC it’s a SoC.
Hi Misha, I understand it is an SoC. Typically what you do as part of SoC designs is license IP (in chip design IP means a blob of VHDL / Verilog code), so let’s say I design an ARM SoC, I get an ARM core from ARM, I may get a video core IP from Imigation, a flash controller IP from someone else. Yes, you can design pieces in-house too. Something like networking is so complicated and I don’t recall AMD having any experience in there, I doubt they started from scratch and just licensed IP from a Marvell or some other company. I’m curious what IP it is unless it is truly their own design.
I found the answer to the AMD NIC question. I noticed the screenshot showed ‘amd-xgbe’ driver, so I looked at the Linux kernel source. Based on the code AMD licensed “Synopsys DWC ETHER XGMAC”, see https://www.synopsys.com/dw/ipdir.php?ds=dwc_ether_xgmac. I don’t know how much this IP does as it provides a MAC layer, so AMD likely had to build some parts themselves, but I’m not sure.
Finally, Patrick, thanks.
I would assume that’s $315 USD in bulk lots of 1000 for the chip? Wholesaler price?
So could we assume at least $550 / $600 US motherboards using this?
I’d still strongly consider it, it certainly seems to perform. I’d prefer perhaps something 15w lighter and $100 lighter, but we’ll have to wait for more boards.
As I posted on the forums, incredibly hugely disappointing that AsRock, SuperMicro, Tyan all either told me “nope, no interest in this!” or flat out didn’t respond!
These would make a monster FreeNAS machine, especially the 3201 model with 8c/8t at only 30Watts!
Patrick, do you know if this problem impacts the Epyc 7xxx or 3xxx series?
https://bugzilla.kernel.org/show_bug.cgi?id=196683
https://forums.freenas.org/index.php?threads/ryzen-stability-on-11-0-u4.59017/
$315 is the tray pricing. Anyone buying these embedded parts is likely buying quantities for motherboard runs.
I believe that kernel bug is for Ryzen users and has not been confirmed on EPYC.
what do you think, is Intel any part of delaying the go-to-market for these chips?
@kpin:
I hadn’t thought of that, it seems unlikely but I wouldn’t put it past them to try that.
I did contact SuperMicro, Tyan and AsRock about this CPU multiple times since Feb and have either had no response or an outright “no” from them, which was disapointing.
However, SuperMicro already do some AMD Epyc stuff, so they aren’t ‘frightened’ of Intel, to my knowledge.
We need a competitive board to the likes of SMCI’s x11sdv-*c-tlnf8 boards. (I hope I got the model name right from memory.)
Would be great if we could get the same treats the Intel line currently has.
Anyone know of a board coming out with the dual 10 Gb option? the supermicro boards which just came out are all 1 Gb.
@Nathan
Asrock Rack EPYC3251D4I-2T supports dual 10 GbE NIC speed (and M.2 slot supports 22110 which is unique to my understanding).
See https://www.asrockrack.com/general/productdetail.asp?Model=EPYC3251D4I-2T#Specifications
cpu socket ???
norbi31 – no CPU socket. These are embedded parts.
Any sign of the higher level SKU 3451, 3401, 3351,3301? that is not COM Express Type 7? And they seem vaporware as well.
https://www.amd.com/system/files/documents/3000-family-product-brief.pdf
Steve! Yes. Stay tuned in a few weeks to STH.
Any updates. This platform looks very cool, and I would like to use it for routers and small appliances, but I can’t find any consumer motherboard or system that actually exposes the built in 10Gbps NICs as SFP+. I did found some boards from Supermicro and ASRock Rack, but they all use Intel NICs for their 10Gbps and 1Gbps ports. Weird. Would be great to see native 4x SFP+ and 2x SFP (via external controller), and this SoC can do it quite well and cost competitively, yet I can’t find products doing that. Weird.
Some information about idle power use to someone like me who uses the platform in residential setting where the system is running 24/7, workload is bursty and idle power use is very important.
With the following configuration I’m getting 22.3W at idle.
M11SDV-8C-LN4F with EPYC 3251.
Stock fan replaced with Revoltec RL036 (mostly due to noise reduction, but likely it helps power use too).
64GB DDR4-2400 RAM (2x32GB HMA84GR7MFR4N-UH)
WD SN750 1TB NVME SSD
Only single gigabit network cable is connected. BMC is left unconnected.
PSU is 120W 12V power brick (Chieftec CDP-120ITX).
VGA is disabled via jumper (saves 1W)
“powertop –auto-tune” has been run (saves 0.2W if VGA is enabled, 1.1W if VGA is disabled)
The measurement has been made on console login screen of stock Debian 11 that had powertop installed. The power meter is not calibrated.