DDR4-2133 versus DDR4-2400 Memory – STREAM benchmarks using Xeon D

A question that is likely to get asked often in the next few weeks is: what is the speed difference between DDR4-2133 and DDR4-2400 RAM. We reported a few months ago that the official specs for some of the newer Xeon D parts listed DDR4-2400 as their supported memory speed, up from DDR4-2133 on the lower end SKUs. In February we had zero systems running DDR4-2400 RAM but now we have about ten with around 1TB of DDR4-2400 deployed so we decided to make a small comparison of two nodes each with different RAM speeds.

Benchmarking DDR4-2133 v. DDR4-2400 using STREAM

For this test we are using the industry standard STREAM benchmark. STREAM is a benchmark that needs virtually no introduction. It is considered by many to be the de facto memory performance benchmark. Authored by John D. McCalpin, Ph.D. it can be found at http://www.cs.virginia.edu/stream/ and is very easy to use.

From a raw MHz gain perspective, we would expect to see a 12.5% increase in performance. This gain is tempered by the increase in latency from 15-15-15 (DDR4-2133) to 17-17-17 (DDR4-2400) on the higher speed modules. Every time we see a DDR frequencey bump, we see the latency figures creep up as well. We ran the STREAM benchmark on each node over a 12 hour period to ensure we had a repeatable result set and we threw out any results that were more than 3 sigma off of the mean (assuming these were benchmark tool anomolies.) As you can see from the above, one can assume approximately a 9.5-10.5% improvement in memory bandwidth for the Xeon D platform, and we would expect from other similar platforms.

Test Configurations

For this test we used 4x 16GB Micron DDR4 RDIMM modules in the most identical platforms we could find, two Supermicro MicroBlade nodes. Unfortunately our larger modules were tied up in other systems:

We utilized the bottom two blades each with an Intel Xeon D-1541 onboard and 4x 480GB Intel DC S3500 480GB drives. You can read more about the Supermicro 3U MicroBlade that fits between 14 and 28 Xeon D nodes in a 3U chassis in our forum thread. We will have a more formal review in the future.

Conclusion

The performance benefit is there and clear when we run a memory specific benchmark. We also ran Linux-Bench on these platforms as well as an older Xeon D-1540 platform and a newer Xeon D-1587 (16 core/ 32 thread platform) and found that overall benchmark suite differences were on the order of 1% better with the faster RAM. For smaller systems like the Xeon D-1500 series where you have limited bandwidth to begin with, the real-world performance impact is not going to be huge. On the other hand, for memory bandwidth sensitive applications, the upgrade, assuming your platform and processor support it, is an excellent one. As we move into 2016 DDR4-2400 is going to be produced on the newest process nodes by DRAM manufacturers which allow them to fit more die on each wafer, driving down production costs. Prices of DDR4-2400 are expected to come down so there is little reason to get DDR4-2133 RAM unless you can get it at a cost savings or if that is the maximum memory speed of your platform. Of course, if you want more DDR4 memory bandwidth and capacity, platforms like the Intel Knights Landing (six channel DDR4) or Intel Xeon E5 series (quad channel RAM) will provide with more aggregate bandwidth. We do hope as the ARM competition becomes sharper in 2016, we hope that the Xeon D’s memory controller gets unleashed to support 64GB RDIMMs (these modules are coming in 2016) and next generations get at least quad channel memory support.