Intel has a new AI training chip, the Gaudi 2. A few years ago Intel Acquired Habana Labs for its AI efforts. That was in the original Gaudi generation. Now, Intel has a new generation of AI training chips called the Gaudi 2. Headlining the feature set is more speed for faster training.
Intel Habana Gaudi2 Launched for Lower-Cost AI Training
The new Intel Gaudi2 is a 7nm OAM module now with 96GB of HBM2E memory. Part of the shrink to 7nm was that the chips got more compute elements and memory. Intel is also using the industry standard OAM form factor instead of a proprietary form factor.
For some sense of performance, Intel says that Gaudi2 is faster than the NVIDIA A100. We will quickly note that Gaudi2 is just launching while NVIDIA is about to launch the H100. We have already had our hands on the NVIDIA H100 in Our First Look.
Beyond moving from 16nm to 7nm, Gaudi2 now moves from 8 in the previous generation to 24 Tensor Processor Cores. Tripling compute resources is a big deal enabled by the newer process. We also have media processing engines added to Gaudi2.
This is important for video analytics workloads, and the inclusion of the hardware accelerator for this is a key enhancement in this generation. As mentioned earlier, not only are compute resources tripled, but onboard memory is tripled to 96GB. SRAM that is on-chip is doubled to 48MB.
Habana’s value proposition extends off chip. The company, prior to its acquisition was founded on using more open technologies. That includes OAM modules as well as using 100GbE RoCE. As you will notice, there are 24x 100GbE RoCE connections per chip, and eight chips in a system. That requires a ton of bandwidth and so there is the OAM baseboard for the chips (we have already looked at the UBB, but Intel’s key partners are Supermicro and DDN here.
Here is the diagram of the platform just to show how it is constructed at the baseboard level.
Compared to NVIDIA that is using SXM and NVLink/ Infiniband, Intel is hoping that using more open/ standard technologies will help to get more market penetration.
Final Words
Intel is one of the few companies that has the resources to challenge NVIDIA in AI. The new Intel Gaudi2 is an interesting response. One item that is interesting here is that this is a PCIe Gen3/ Gen4 devices instead of a Gen5 card. That means Intel is pairing the new accelerators with Ice Lake instead of the Sapphire Rapids generation.
Open standards are a good thing. I hope this helps Intel better compete with nVidia’s offerings. As always, pricing without discounts is the big question. I hope can price this competitively against the upcoming H100 and convince me that their oneAPI is worth investing in.