The PCIe Gen4 era is upon us, and storage is set for its first major performance bump in years. Leading this new era is the Kioxia CD6 and CM6 NVMe SSDs. These are 2.5″ PCIe Gen4 U.3 SSDs that are capable of pushing 6-7GB/s of sequential read performance and 1M+ random read 4K IOPS.
Why PCIe Gen4 is Going to Invigorate the Storage Market
Kioxia is announcing two new PCIe Gen4 NVMe 1.4 SSDs. These are the first generally available PCIe Gen4 2.5″ U.3 NVMe SSDs on the market.
For a quick showcase of the impact, Kioxia noted the Microsoft Ignite 2019 demo with QCT, AMD, Broadcom, and Kioxia where Microsoft was able to use 8x Kioxia CM6 PCIe Gen4 7.68TB NVMe SSDs to hit 52.1GiB/s speeds with two nodes and four drives per node or over 26GiB/s with four CM6 drives per AMD EPYC 7002 node. You can read more about QCT’s AMD EPYC Rome servers in our piece Quanta AMD EPYC Rome Servers Set to Make a Splash.
In terms of generational improvements, if you use Intel Xeon Scalable servers, you will likely order Kioxia CM5 SSDs. If you are on AMD EPYC 7002 series, you can order Kioxia CM6 at essentially the same cost and see enormous performance improvements.
Kioxia is also segmenting its PCIe Gen4 launch set with two types of drives. The first is what it calls “Enterprise NVMe SSDs” and the second it calls “Data Center NVMe SSDs”. The idea here is that the Kioxia CM6 is designed for single and dual-port workloads with more enterprise-type features to compete with SAS arrays in the enterprise (think Dell EMC machines.) On the other side, the Kioxia CD6 is designed to focus more on read workloads and cost optimization which is what cloud buyers want to see. Here, Kioxia is focused on providing a SATA alternative, albeit with closer to 10x the performance at a similar price per TB.
We are not going to re-hash what is on Kioxia’s slide, but the drives can range from 800GB to 12.8TB on the mixed-use side with 3DWPD and more overprovisioning. The Kioxia CD6 also offers 960GB to 15.36TB 1DWPD drives. Specs vary a lot by drive capacity but can hit 6.2GB/s reads, 4GB/s writes, and 1 million 4K random read IOPS.
For the enterprise market, the Kioxia CM6 provides capacities ranging from 800GB-12.8TB (3DWPD) to 960GB-30.72TB (1DWPD) and even more performance than with CD6. These drives can hit 6.9GB/s officially, but Microsoft saw better results noted above. They can also hit 4.2GB/s writes and 1.4M random read IOPS. In the days of SATA SSDs, we were excited to see entire arrays of 24x SATA III SSDs hit 1M to 2M read IOPS. Now, single drives are squarely in that range.
You may have noticed that Kioxia is featuring NVMe 1.4. That is a new revision of the NVMe spec that the company is supporting. The company highlights some of the new enhancements in this spec such as shared stream write, persistent event log, sanitize enhancements, and namespace granularity.
Final Words
Storage has been relatively boring for a few years now. Once SSD vendors were able to saturate the PCIe Gen3 x4 bus, we saw mostly feature expansion and some feature expansion, but relatively modest top-line performance gains. With PCIe Gen4, SSD vendors now have the opportunity to innovate potentially doubling performance and lowering latency.
The reason this has been so slow is that Intel Ice Lake Xeons are likely going to be, at the earliest, later in the second half 2020 for anyone other than large cloud customers. That is Intel’s leading PCIe Gen4 data center platform. For now, you will need an AMD EPYC 7002 Series “Rome” platform designed for PCIe Gen4 storage. As an example, the Gigabyte R272-Z32 has PCIe Gen4 slots but is wired for PCIe Gen3 drives. If you want to take advantage of new drives like the Kioxia CD6 and CM6, you will want a PCIe Gen4 (storage) capable platform and AMD EPYC 7002 series processor. That is still a smaller segment of the market. If you can drive value from faster storage, PCIe Gen4 offers faster drive connectivity as well as the opportunity to use PCIe Gen4 x16 200GbE network adapters. Kioxia has an offering that offers something completely different in the market until other vendors catch up.
What is Kioxia?
Kioxia is the new name of Toshiba Memory. We covered this in Toshiba Memory is Dead Long Live Kioxia Holdings. If you previously purchased Toshiba SSDs, these are now part of Kioxia’s lines. Since the name is not easy for some to pronounce, we have a How to Pronounce Kioxia guide:
Some are still using strange pronunciations, so use this video to win a lunch bet off of co-workers.
So what servers have PCIe 4.0 capable hotswap backplanes at this point?
These are VERY exciting to me for Data Warehouse servers. I just ordered my first AMD EPYC 7002 server for testing, and I’d love to stuff it with eight of these. So who will take my money, and when can I buy?
Gigabyte has a nice AMD-EPYC-7002 portfolio.
I still think it is an Aztec plot.
Any idea, when these drives will be on the vmware vsan hcl?
Any power consumption numbers? And how would these fare in a PCIe3.0x4 environment, in case you’d want to use these drives while waiting for a PCIe4.0 capable server?
Any news on the availability? No other product like that (Enterprise level u.3 NVMe Pcie 4) is available yet on the market as of May 2020.
Yeah, keen on some info on availability as well.