At OCP Summit 2023, we saw the ASUS RS720QN-E11-RS24U. This is the company’s high-density NVIDIA Grace Superchip Arm server that can fit 576 Arm Neoverse V2 cores into a 2U chassis. Since this is a bit different, it is worth showing off.
ASUS RS720QN-E11-RS24U NVIDIA Grace Superchip Arm Server
Taking a look at the rear of the server, one can see two power supplies and four sleds for systems. The power supplies in this system are 3kW-3.6kW because of how much power the Grace Superchip modules use (500-600W each.)
Here is a node that we pulled out at the show.
One can see that the NVIDIA Grace Superchip module area spans most of the sled.
Here is the area from another angle. Something to keep in mind here is that the module has both the two 72 Arm core chips as well as the LPDDR5 memory, so this area is equivalent to a dual-socket solution with memory slots around it.
The small riser to the side of the socket is a dual PCIe Gen5 x4 M.2 (22110 110mm) module for boot drives.
Here, we can see the ASPEED BMC along with the other functions such as onboard LAN. There is also a standard OCP NIC 3.0 slot. The risers have two options. One can either have a full-height single slot x16 card for something like a NVIDIA BlueField-3 DPU. The other alternative is two low-profile x16 cards. All of the expansion is PCIe Gen5.
Here are the high-density chassis connectors.
One item to note is that some 2U 4-node (2U4N) systems use fans that are on sleds for easy service. Others, like this ASUS, use midplane fans for better power consumption characteristics. ASUS also has a liquid cooling option for the CPUs using cold plates.
Final Words
Overall, it was cool to see this system at the show. The NVIDIA Grace Superchip effectively has a dual 72-core CPU and memory on a single module, so this should be interesting. We are hearing availability will start ramping as these CPUs have been challenging to come by at this point as NVIDIA has focused so much attention on its data center GPU business. It is really cool to see not just single module and dual module NVIDIA MGX servers, but also 2U4N designs like this one from ASUS hit the market.
