AI is Rocking 5 Key Takeaways from NVIDIA GTC 2024

This week I was at NVIDIA GTC in San Jose, California. It was crazy. The last big GTC was in 2019, just as NVIDIA was completing its Mellanox acquisition. This year, that acquisition turned into a key component of its new DGX systems. The bigger takeaway though, was that GTC was rocking. That is said coming from a perspective of having been to hundreds of industry shows over the years. As such, we have five key takeaways from the event.

AI is Rocking 5 Key Takeaways from NVIDIA GTC 2024

Here are the top 5 key takeaways from GTC 2024:

• One brutally apparent observation is that AI is not a part of HPC. GTC 2024 had over 17,000 people. Folks were excited about SC23 hitting 14,295 attendees in Denver, and the thought was that was swelling because of AI. Of course, both shows have off-site, non-badged attendees doing briefings, showing NDA products, and closing deals. At the same time, GTC felt full, even on the FOURTH day when usually show floors are barren. Here is a look at the hallway outside of the show floor after it closed, and that was normal traffic to see during the four day event.

• Develop power systems today. The NVIDIA DGX GB200 NVL72 is a mouthful, but it is also a powerful system that uses 120kW. Eight of those, plus necessary storage, switching, and so forth is likely over 1MW of power in a single data center aisle. Liquid cooling is going to be important. We have looked at generations now of top-end NVIDIA liquid cooled servers. Fluid, fittings, metal blocks, and tubes are not the constraints on the future of compute. Power is. Most customers today can figure out 30kW racks. 60kW racks are doable in many data centers. 120kW is an enormous jump. At the scale of many megawatts, just sourcing enough low-cost and reliable power is a global need.

• NVIDIA is not a gaming company. We discuss NVIDIA GPUs as graphics processing units. The heritage was, without a doubt, the GeForce (and Riva?) lines, professional graphics lines, and so forth. GTC 2024 talked a bit about OVX and Omniverse. The focus was clearly on data center AI (Blackwell), networking (NVLink/ NVSwitch and ConnectX-8 Infiniband), and Grace. Robotics (Jetson and Drive) were certainly there. The keynote, show floor, and so forth were dominated by data center, robotics, and automotive. GTC has always had a slant in this direction but after two days I was reflecting that if I did not use NVIDIA GPUs regularly and this was my first GTC, I would have almost no idea that NVIDIA sells cards for gaming after two days in the conference.

• Robotics is the dark horse. NVIDIA might talk a lot about AI today, but one of its big bets for the future is that the same LLM technology used to flood the world with text, images, and, increasingly, video is going to make its way to robotics. Encoding movement and more into LLMs sounds like a promising way to more efficiently program robots. If we want Rosie the Robot from The Jetsons, or something even more human-like, hardware that can take advantage of the massive data center investments in LLMs is required. Jetson Thor might be a big one when it launches in the first half of 2025. Until then, we have Jetson Nano.

• NVIDIA has a wild set of interconnects. Just keeping track of these is challenging. For scale out there is Infiniband. For management, Ethernet. GPU-to-GPU is NVLink. GPU-to- Grace CPU is Chip-2-Chip (C2C). Blackwell half to Blackwell half is via NV-HBI. NVIDIA is designing a huge number of interconnects just to move data around. That is something that a lot of folks forget when looking at accelerator FLOPS/ TFLOPS. A startup not only needs to get the on-package, and in-chassis right, but it needs the scale-out fabrics as well.

Final Words

Everyone I spoke to at NVIDIA GTC 2024 commented about how busy it was. NVIDIA had an employee hour at the end which was great. There were many green badges walking around the show floor and a few stopped to say hi, take selfies, and then talk about what they make, often pointing to the product.

Here is a better way to put it. I spent four days and an evening at GTC, and I never got to probably 20% of the show floor nor a few exhibits I wanted to see. Normally, I get at least an hour to myself after meetings and already covered most of the floor. Not this trip. GTC was rocking.