NVIDIA RTX 5000 Ada Compute Benchmarks
At STH, we do not test games on GPUs, on either professional or even gaming-focused products. We also took a pause in the series when GPUs spiked in price and had tight availability over the past few years. Now that GPUs are a more regular market, we are rebooting the series, including adding new benchmarks.
We are going to cut down on some of the results to streamline the process for folks as we have gotten the feedback that we have too many charts.
Geekbench 5 and 6
Geekbench 5 and 6 measure the compute performance of your GPU using image processing computer vision to number crunching.
The top result is Geekbench 5, while all below are from Geekbench 5 runs.
Geekbench 6 was released this year. It does not include CUDA benchmarks which I hope they bring back. Instead, they use OpenCL and Vulkan.
This is going to be a theme we are going to see quite a lot in this review. The RTX 5000 Ada is a notable step down from the RTX 6000 Ada.
LuxMark
LuxMark is an OpenCL benchmark tool based on LuxRender.
This is one where we can see why many of NVIDIA’s and PNY’s customers would choose the RTX 5000 Ada. The performance is still excellent and the incremental price/ performance of going up the stack is not necessarily there.
hashcat64
hashcat64 is a password-cracking benchmark that can run many different algorithms. We used the Windows version and a simple command of hashcat64 -b. Out of these results, we used five results in the graph.
Here the consumer GPUs tend to do very well. That makes sense, given the needs of these types of workloads. Still, there is a benefit to the newer generation of cards.
I thought gaming GPUs didn’t use blowers because of an enforced market segmentation by Nvidia to prevent those GPUs from being used in the data center.
Yes. The dual slot blower is officially banned from nvidia for top-end cards. However we still have vendors making them and offering it as “cheap alternatives to dc cards”. This is something Nvidia does not want OEMs to make, but unable to ban OEM from making because the potential market is just so high.
The article doesn’t mention it, but I see these have been crippled when it comes to dual precision compute, just like the previous generations.
Ada, Ampere, Turing – no DP capable cards (or at least not severely cut down to 1/64 of SP performance).
Volta (GV100) is the last DP compute card released, and that’s becoming a bit outdated.
Once upon a time, these Quadros (or whatever they are named today) were engineer’s cards. You had to have a high-end one to accelerate engineering simulations (FE, CFD and similar). I guess AI stole the show, and nobody is going to cater to that small market anymore.
What’s the proper way to build a DP crunching workstation nowadays, anyway?
A key reason not discussed in the review for why some users will almost have to use this card vs a much cheaper and faster 4090 is the ECC Memory this card has. For some applications and uses, that is required, also for liability reasons.
Apart from all that, I prefer blower cards if available; unfortunately, current generation consumer cards in that design are almost impossible to find.
Actually you can enable ECC on 4090 the same way on RTX workstation cards. These cards both lose a portion of the total VRAM capacity if ECC is enabled, 4090 just has half of the total capacity.
@TurboFEM: That’s because both NVIDIA and AMD have abandoned FP64 in mainstream architectures as its market share is not worth the cost of silicon to implement it at full speed. Gaming doesn’t need it, and neither does AI/ML.
On NVIDIA side the H100 can perform one FP64 operation every 2 cycles, while the Ada can do it every only every 4 cycles. AMD has implemented native FP64 since CDNA 2, and further improved it in CDNA 3.
So basically for FP64 you need to go for the highest end compute accelerators.
@eastcoast_pete: There’s also driver support and qualifications that are critical for certain uses. Using mainstream cards and mainstream drivers is out of the question for them.
@Gnattu: NVIDIA specifically forbids using mainstream cards in commercial compute (via CUDA and driver EULAs), and actively goes after companies who, for example, rent them as public clouds. While you can try to use them internally, your legal department won’t be happy if they ever find out.
I’m glad to see that others have already mentioned blower GPUs didn’t fall out of favor with consumers, Nvidia enforced that consumer AIBs couldn’t use blowers to ensure that the cheaper RTX3090/4090 wouldn’t be used in workstations instead of their astronomically priced workstation GPUs. I see it’s very popular right now by blogs of all types to gloss over Nvidia’s hostile behavior towards consumers but it’s a damn shame.
Thanks John. That is good info as I am weighing upgrading from A6000 cards.
Running large language models is becoming increasingly common. Suggest having a benchmark for that in the future. For example running the 8x7b mixtral model is common these days.
@Kyle Actually I know some companies get caught. The nvidia geforce driver has its own telemetry so Nvidia knows what you are doing if you don’t cut this connection. The result? Companies now start to disable public Internet access for these nodes and you have to distribute work through a gateway so that the telemetry never reaches Nvidia. I know it is prohibited by Nvidia, but the amount of money we are talking about here is unlikely to be limited by an EULA.
@Gnattu: Oh sure you can work around this issue for internal use. The problem is when you try to sell it to the public as a cloud offering, for example. You can’t really hide the fact you’re using a consumer GPU then – your clients will be able to tell. The issue is whether those clients will care.
EULAs in general are a murky topic, but most “serious” companies will not even try to get into the grey zones.