I have probably taken apart over two dozen GPU servers over the years. I have taken apart at least two AMD Instinct MI300X / MI325X systems this week in Taipei, plus a few other fun servers. I finally asked a question that I felt ashamed to simply ask. What is this thing on these big GPU baseboards?
This is a Fun AMD Instinct MI325X 8-GPU Baseboard Feature
As a quick refresher, the AMD Instinct MI300X was the 192GB HBM3 memory GPU that comes in an OAM form factor. Eight of these come on a Universal Baseboard (UBB), and that assembly is roughly in line with a NVIDIA HGX 8 GPU platform, except NVIDIA has the NVLink Switch architecture on its HGX 8 GPU platforms.
The AMD Instinct MI325X is the HBM3e update with 256GB of HBM3e memory for higher capacitity and more memory bandwidth. It comes in a very similar platform. In the front of the platform’s massive heatsink array there is a big black metal piece.
Here is the other side where there is another one of these metal towers fitting around the PCIe retimer heatsink.
We had to swap an AMD Instinct MI325X 8-GPU baseboard today and these things are heavy. The shipping box with foam is around 39.5kg or 87lbs. Those black metal loops I was told are actually handles to simply help move the giant assembly.
After being left in the dust by a gentleman who used these handles and replaced the entire UBB in under 90 seconds, I just had to double-check that I had seen these in systems before and never knew. Here is a fairly clear photo from the 2021 Inspur NF5488A5 8x NVIDIA A100 HGX Platform Review and we can see the metal handle here as well.
The funny part is that I have seen these handles for years and across many systems, and just never knew what they were.
Final Words
Modern 8-GPU assemblies from AMD and NVIDIA are heavy because they require a lot of metal in their heatsinks. Even the liquid cooled coldplate versions may look smaller, but the copper blocks are often heavy. Having these handles makes handling these expensive baseboards much easier.
Maybe in the new year there is a key takeaway here: just ask a question even if it seems silly. The purpose of this post is that if I had the question, then there is a good chance someone else has a similar one.
