Aivres KR6288 NVIDIA HGX H200 Server Review

3

Moving to the rear, a lot is going on.

Aivres KR6288 Rear
Aivres KR6288 Rear

In the top left, there are four low-profile PCIe Gen5 x16 card slots. Here, we have a set of four 400G NVIDIA ConnectX-7 InfiniBand cards.

Aivres KR6288 NVIDIA ConnectX 7 400G Left Rear
Aivres KR6288 NVIDIA ConnectX 7 400G Left Rear

In the center section, we get full-height card slots and our rear I/O. We have an NVIDIA BlueField-3 DPU for our north-south network installed here. For rear I/O, there is a management interface, two USB 3 ports, and a VGA port. Again, most will want front-panel KVM access instead.

Aivres KR6288 Rear IO And Center Expansion
Aivres KR6288 Rear IO And Center Expansion

You can see that the I/O tray is removable via the two levers/ latches below the rear I/O block.

On the right, we get four more low-profile card slots filled with NVIDIA ConnectX-7 NICs. It is common to see one 400G NIC per GPU these days for east-west traffic and another 400G NIC, often a BlueField-3 DPU, for north-south networks.

Aivres KR6288 NVIDIA ConnectX 7 400G Right Rear
Aivres KR6288 NVIDIA ConnectX 7 400G Right Rear

The bottom rear of the server is all fans, although many of these fans are installed in power supplies.

Aivres KR6288 Rear Fans And PSUs
Aivres KR6288 Rear Fans And PSUs

We have started seeing two different types of power supplies used. One set is for the 12V output required for the CPUs, motherboard, NICs, and so forth—perhaps better stated for all of the devices in standard servers. The other set is for the 54V output for the GPUs. It is becoming more common to see server designs with two types of power supplies like this.

Aivres KR6288 Left Rear PSU Markings
Aivres KR6288 Left Rear PSU Markings

When installed, the 3.2kW 12V power supply is on top, with the three 2.7kW 54V PSUs below.

Aivres KR6288 Left Rear PSUs
Aivres KR6288 Left Rear PSUs

The same setup exists on the right rear as well.

Aivres KR6288 Right Rear PSU Markings
Aivres KR6288 Right Rear PSU Markings

Here are those power supplies installed.

Aivres KR6288 Right Rear PSUs
Aivres KR6288 Right Rear PSUs

These six dual fan modules for the main fans cool the NVIDIA HGX H200 8-GPU assembly.

Aivres KR6288 Fan Modules
Aivres KR6288 Fan Modules

Here is a look at the midplane from the rear, where the power supplies and fans plug into.

Aivres KR6288 Midplane Fan And PSU Rear View
Aivres KR6288 Midplane Fan And PSU Rear View

Here, we can see the other side of that middle airflow pathway again, read the power meter, and see the Kioxia CM7 SSD.

Aivres KR6288 Rear See Through To Power
Aivres KR6288 Rear See Through To Power

Next, let us get to the HGX H200 8 GPU assembly.

3 COMMENTS

  1. “designed to house two processors, 32 DIMMs, nine or more 400G NICs, and eight GPUs with over 1.1GB of combined HBM3e memory.” Minor typo at the beginning, I think that you mean tb ;)

  2. Inspur was forced to exit since if they owned a company they couldn’t have a server with H200’s. Kaytus was the one who went to Singapore? Aivres was spun out as the US operations and sales as its own OEM. If they were a Chinese brand owned by Inspur they couldn’t get the H200’s for this server. I’m seeing a H200 server from them, and their business addresses are all in CA, so I don’t think it’s Inspur

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.