AMD EPYC 7351P Power Consumption
The other side of the equation is power consumption. The AMD EPYC 7351P is putting up some impressive benchmark numbers, but that does have an associated cost. Here is what we saw on our PDU after a few runs:
Idle power consumption was around 75W measured on the 208V PDU at 17.7C and 71% RH. Our testing window shown here had a +/- 0.3C and +/- 2% RH variance.
In terms of power consumption, and again this is without 24x NVMe drives installed, this is an idle result comparable to Xeon Silver. The maximum power consumption and what we are showing as a 70% load are both significantly higher than a single socket Xeon Silver or a dual Xeon Silver 4110 system by a wide margin. The dual Intel Xeon Silver 4110 setup will not hit above 200W running the same workloads as an example.
AMD is delivering a lot of performance in the sub $1000 segment. At the same time, that comes at the expense of power consumption. If you are saving $1000 on CPUs and systems and spending $15/ mo for another .12kW, that is a fairly easy TCO calculation to do.
Market Positioning
As we pointed out in our dual AMD EPYC 7251 review, AMD has a number of competitive vectors. The obvious competition is the Intel Xeon single socket line. As a P series part, AMD is also positioning the CPU against lower-end dual Intel CPU configurations. We also see some competition from the dual AMD EPYC 7251 and the AMD EPYC CPUs around the same price range.
AMD EPYC 7351P v. Intel Xeon Silver
Since the AMD EPYC 7351P is a $750 part (although we are seeing street pricing slightly above that at the time of publishing around $820), it is not competing directly with Intel Gold CPUs, instead, it is competing with the Xeon Silver range. With the Intel Xeon Silver 4108 priced at just over $400, we see the EPYC 7351P priced as a significant competitor to that configuration. Likewise, the Intel Xeon Silver 4114 is a similarly priced single-socket Intel Xeon Silver option.
On the question of Intel Xeon Silver 1P and 2P configurations versus the 7351P, there are a few main points to consider. First, AMD flat out has more PCIe lanes with 128. Intel has 48 on the Intel Xeon Silver, although in dual socket configurations (96x) you could argue using a PCH does bestow some benefit in using those lanes efficiently. Second is memory capacity. The AMD EPYC 7351P can handle 16x DDR4-2666 DIMMs (8 channel) and up to 2TB of memory. Intel Xeon Silver can handle up to 12x DDR4-2400 DIMMs and 768GB of memory in single socket mode and twice that in dual socket configurations. Third is in terms of platform. Intel Xeon Scalable has more platform options available, and there are a few features that are more mature on the Xeon Scalable platform (e.g. NVMe hot-plug/ swap and QuickAssist) that AMD platforms still have to catch-up on in terms of ecosystem maturity.
If you run Dockerized microservices and are not leveraging features such as QAT, EPYC simply has Intel Xeon Silver beat. Likewise, as in the case with this Tyan test system, AMD having more PCIe lanes makes a compelling argument. It is not that Intel does not have parts that compete with AMD. Intel’s product segmentation including features and pricing where features like dual AVX-512 are not present on low-end SKUs, DDR4-2666 and higher memory capacities are not supported on Xeon Silver. Omni-Path is not supported on Xeon Silver (as on package fabric.) In terms of volume, the Intel Xeon Bronze and Silver are low value but high volume parts and Intel needs to do some soul-searching. Intel Xeon Silver does have better power consumption characteristics which is important yet we have seen IT buyers accept higher power consumption for higher performance.
What AMD essentially did with the EPYC 7351P is put forward a high-performance offering in a price segment where Intel is geared specifically for low power use. To be clear, Intel does have high-performance chips. The sub $1000 CPU segment for Intel is focused solely on low power consumption, not performance.
In 2017/ 2018 it may be enough that Intel is Intel and has commanded huge market share for years. In upcoming generations, that is going to be a harder sell. Either way, if you have a dev ops team EPYC should be a part of this year’s server purchases.
AMD EPYC 7351P v. AMD EPYC
In the x86 market, the AMD EPYC 7351P is not just competing against Intel, but also other SKUs in the AMD stack. Two AMD EPYC 7251’s are slightly more than a single AMD EPYC 7351P. The ability to use more memory per system or more lower capacity and less costly DIMMs may make that a compelling solution for buyers.
Unlike many of the other SKUs, as a “P” series part, the AMD EPYC 7351P sits between the EPYC 7281 and EPYC 7301 pricing wise. For us, the choice is clear, if you are using a single socket system, get the P series EPYC parts in that price range. A good debate when purchasing this is whether or not to move up to the EPYC 7401P since one gets 50% more cores at slightly lower clock speeds for a relatively small price premium. We are going to have more on that soon, but essentially moving one “P” part up the stack pits a $1075 AMD CPU with 24 cores against the 12 core Xeon Silver 4116 at around $1020. While Intel may have some IPC advantages in some workloads, the de-featuring of the Xeon Silver line makes AMD’s offerings really shine even more so than we are already seeing with the EPYC 7351P.
Final Words
At STH, our first introduction to AMD EPYC was the dual EPYC 7601 configuration. Our test system originally had an AMD EPYC 7601 installed and we were planning on first publishing 1P 7601 results. As we worked through various other configurations, it became clear that while AMD is very competitive at the high-end, its mainstream offerings are competing with de-featured Xeon Silver CPUs and absolutely obliterate what Intel is offering. Addressing the obvious, one has to be willing to be different at their job and deviate from purchasing Intel x86 to buy AMD instead. Here one can take solace in the fact that the AMD EPYC 7351P is not marginally better than the Intel Xeon Silver single (and low-end dual) socket offerings, but instead, there is an enormous chasm created by Intel’s conservative product offerings in the Xeon Silver space versus AMD’s aggressive moves in the mainstream market with the EPYC 7351P. While the EPYC 7351P looks impressive compared to Xeon Silver, we are soon going to show the EPYC 7401(P) numbers that will make you pause.
We also run our own hosting cluster that is currently all Intel-based. AMD EPYC platform maturity and our experience with the Tyan Transport SX TN70A-B8026 thus far means we will be adding AMD EPYC in the next round of upgrades in Q4 2017/ Q1 2018. AMD systems and CPUs are starting to hit the channel for smaller buyers like us. We now have systems from several different vendors already up and running in the lab and the overall platform maturity has come a long way since June.
Great review.
Looking forward to a comparison between AVX512 and GPU.
GROMACS would be a perfect benchmark. When still availible with this very nice system https://www.servethehome.com/tyan-thunder-hx-ga88-b5631-server-review-4x-gpu-1u/
and/or DeepLearning 10/11
And again, great review.
I’m in IT procurement for a big F100 company in the US. We’re struggling with AMD to Intel now and our IT staff is trying to quantify for the business case. You may have done their jobs for them. +1+1 for the timely article. I’m emailing them this now
Great job AMD team.
Just what I needed. Your point on completeness is good. It’s also good that you’ve got a consistent setup and enviro for testing. We use SPEC06 for our purchases but it’s tough to use. You know it’s the OEM with the best tuning team getting the best results. I’m also sure that the OEMs have some special cooling that they’re using or something for their runs to get the best #’s. Having AMD and Intel H2H is a value to the IT community. Scripted + same rack = consistent. We know we’re buying nodes. This is helpful for gauging relative performance improvement.
3 ?’s:
— Ever consider using less stodgy lingo? It sounds like you’re a professor.
— You mention a larger data set. I can see you’re running more than you’re reporting here. Are you accepting inquiries to purchase an expanded set?
— Are these nodes going on DemoEval?
So for power draw.
7351P max load 340W?
Dual Xeon Silver 4110 under 200W?
The 2 seem to have same performance on some tests.
Is that correct?
Thank you.
@patrick Do you know if an Epyc P part will run fine as the single processor in a dual socket board? Or it must be installed on a single socket board?
I would like to see how these perform for workloads like medium sized databases (postgres / MySQL) compared to Intel, I suppose these kinds of workloads may suffer a lot more from NUMA than the ones in your testsuite.
Prosgres largely disk io bound, verdad?
That Tyan server looks amazing with all that NVMe connectivity, and the BCM5720 NIC with dual 1G RJ45. We are using the Silver 4110 in a lot of configurations, so I can see the EPYC P-processors as a good alternative, offering greater value, performance and connectivity. Often I go for single-socket, which is advantageous from a licensing perspective, but it often means a compromise on I/O.
From the article is mentioned “The AMD EPYC 7351P can handle 16x DDR4-2666 DIMMs (8 channel)”. In the presentations I have seen, EPYC only support DDR4-2666 with 1 DPC, and DDR4-2133 with 2 DPC. Has this been improved with microcode improvements similar to desktop Ryzen?
Could you post some benchmarks under Windows 10 Pro / Windows Server like Cinebench or FFmpeg? I think EPYC processors can be very useful for 3d content production and multimedia production. Thanks in advance and congrats for the reviews!
@Francesco F
I second that, but understand that Servethehome is focused on server benchmarks.
And yes EPYC is a wonderfull platform for a program like Davinci Resolve which runs mainly on GPU.
Staxrip is a good program to see how the multi-thread speed is (lot better than handbrake).
A stellar article read this morning. Thanks for doing Xeon D as well. Xeon D costs more for 16c but it’s got a miniscule tdp of 65w for 16c. Intel really needs some cheaper performance parts in the silver line. I’m all for low power but there’s no performance option in Silver or Bronze.
We don’t need every number but a good set of relative numbers is all we need. This hit the spot. I don’t like seeing places use geekbench or others with user submittals since ya never know how differ to the systems being tested are or if there’s something else running.
@Misha Engel
I understand, but some software like V-Ray (Cinebench test), Adobe Media Encoder (FFmpeg test) works primarily on CPU. STH have already done some Cinebench test in the past.
+Franchesco F don’t they have c-ray here and show Cinebench can’t handle high spec nodes in that linked article? Cinebench is worthless on servers now. I’m reading the words in the article and not comprehending your comment? Or you just want to see Cinebench and Windows on a CPU where Windows 10 Pro will occupy 0.00000000001% of the install base?
I’d like 24c results.
@WombatFool, so the guys that run RebusFarm (https://us.rebusfarm.net/en/company/render-farm-specs) or RenderRocket or FoxRenderFarm or Rescale or… they are just a bunch of fools? The 0.00000000001% as you said? I’ve just asked for one additional test, if it isn’t possible there is no problem at all.
Seems my reply get stuck, anyway well done, nice benchmarks!
Over at phoronix they also did some tests with numactl which essentially took a single 7601 epyc ahead of a dual 6137 gold server, which is quite impressive.
I really had high hopes for the new xeons but cutting away the avx units drove me to order some new broadwell nodes.
As soon as there is a quad node in 2U server for epyc available, we will switch to epyc.
@Tobias
As soon as there is a quad node in 2U server for epyc available, we will switch to epyc.
Hope AMD will give permission to the MOBO makers. Looking at this article https://www.servethehome.com/amd-epyc-infinity-fabric-update-mcm-cost-savings/ there is no reason not to come up with quad node version.
Wow this is the best. you’ve got so much comparison data for a site that isn’t aggregating user data. i’m seeing like every server cpu combo under $1K. i like the c3958 and c3955 and d-1567 too for other 16 core intel. it appears cheap by comparison. good job amd too on delivering cheap parts with cores and performance
@Misha
it seems like SuperMicro is working on something: H11DST-B
But it is too late for us this year. What was nice, even our distributor preferred the broadwell over the silver cpus.
Do you happen to know a good MKL alternative? I know ATLAS and openBLAS, but had no time to test it against MKL, the biggest problem will be a good fortran compiler, I guess IFORT still beats gfortran by a large margin
@Tobias
Thanks for the tip.
Fortran is a long time ago for me so can’t help you with that.
@Nino
It depends mostly on the size of the data set. If it fits completely in RAM then only writes are i/o bound, although this might be interesting by itself since of course I/O also suffers from NUMA, especially with fast NVMe drives.