GPU Benchmarks on STAR-CCM+ with ATA & Exxact

Harnessing the Power of GPUs for CFD Simulations

In Computational Fluid Dynamics (CFD), speed and accuracy are paramount. Engineers often face the challenge of reducing the time it takes to solve simulations without compromising the fidelity of their results. In Siemens STAR-CCM+, engineers are adopting GPUs to accelerate their simulation times.

Exxact and ATA collaborated to run benchmarks highlighting how GPUs can dramatically outperform traditional CPU-only setups in solving complex CFD problems in Siemens STAR-CCM+, bringing both time and resource efficiency to the forefront.

Why GPU Acceleration Matters for Engineers

The evolution of GPU computing has reached a point where it not only complements CPU capabilities but, in many cases, surpasses them. For engineers and researchers, this means the ability to iterate faster, model larger systems, and tackle challenges that were previously infeasible.

• Time: Faster time to completion results in the ability to iterate and fine tune the model prior to production. Faster completion also means less time spent waiting for results, accelerating the workflow process.
• Complexity: Increased speedup even on highly complex models enable better representations of the simulation. Convergence rates comparing CPU and GPU are negligible allowing engineers to remain highly confident when solving the dramatically faster GPU.
• Cost: A single NVIDIA RTX 6000 Ada has equivalent performance of 250 CPU cores but can be configured in a single workstation or server. 250 CPU cores require a multi-node setup with fast interconnect, increasing cost to run and maintain. Even 8x NVIDIA H100s can be configured in a single server but performs equivalent to 3,000 CPU cores.

With tools like NVIDIA RTX 6000 Ada and H100 GPUs paired with software like STAR-CCM+, the future of CFD looks faster and more efficient than ever. Let's dive into these benchmark results to explore the capabilities of modern GPUs in accelerating CFD workflows.

Graph 1: Speedup for 20,000 Iterations on Various Workloads

The first graph showcases relative speedup when solving CFD cases for 20,000 iterations across different scenarios, comparing an NVIDIA RTX 6000 Ada GPU to a CPU-only setup using an AMD EPYC 9654 96-core processor.

Some of these models you may be familiar with but ATA has increased the complexity and refined the models to better saturate the 96-core CPU to obtain a more representative performance speedup in more moderate-sized use cases. ATA engineers also note that there were some minor modifications to the use cases to make them fully GPU-compatible, otherwise would be kicked back to CPU-only solve:

• The hypersonic sphere's traditional AMR has been removed and uses a pre-adapted mesh instead
• GPU cooling tutorial's orthotropic material for the PCB has been changed to isotropic

You can view ATA's engineer explaining the use case during this webinar they hosted featuring Exxact:

These benchmarks, conducted on Exxact hardware, affirm the immense potential of GPUs in single-workstation setups, delivering results over three times faster than a high-end 96-core CPU.

Utilizing just a single NVIDIA RTX 6000 Ada can accelerate the time to completion for small engineering simulations. We want to emphasize that 96 CPU cores is a substantial amount of computing. However, even with the high core count, GPUs still offer increased performance in various simulation test cases. Additional RTX 6000 Ada GPUs can further increase performance capabilities in the same workstation form factor.

Graph 2: NASA High-Lift CRM – GPU Scalability

The second graph takes scalability a step further, illustrating performance gains in a server deployment featuring up to 8 NVIDIA H100 and 8 NVIDIA A100 GPUs in the same use case of drastically different sizes:

• Case 1: Large Model (57M Cells) in Dark blue
• Case 2: Huge Model (106M Cells) in Light Blue

Running these tests and seeing these performance speedups tell a compelling story comparing GPUs and CPUs and the viability of CPU only deployments in highly complex scenarios.

• In Case 1 with a 57M cell count model, 8 NVIDIA A100 GPUs yield a 14.29x speedup, while 8 NVIDIA H100 GPUs yield a 20x speedup compared to a 96-core CPU, drastically accelerating the time to completion for unfeasible simulation tasks.
• Across Case 1 and Case 2, we can see that the larger the model size gets, the more compelling GPU accelerated computing becomes delivering an even faster performance speedup. 8 NVIDIA A100 GPUs yield a 16.67x speedup, while 8 NVIDIA H100 GPUs yield a staggering 25x speedup.

These tests—conducted by Siemens using STAR-CCM+—demonstrate the software's ability to effectively leverage multi-GPU configurations, enabling engineers to simulate large and complex CFD models in record time. To attain the same performance as the 8 GPU compute nodes, a deployment of multiple CPU nodes totaling over 2000 CPU cores would be required. The cost continues to add up when accounting for additional power, rack space, cooling requirements, networking, and more. When transitioning from CPU to GPU-accelerated computing, businesses save time and money.

Key Benchmark Takeaways

1. Single-GPU Advantage:
   • For smaller workloads or individual engineers, a single RTX 6000 Ada GPU can dramatically reduce simulation times compared to even the most powerful CPUs.
   • This makes GPUs ideal for iterative design processes or preliminary analyses.
   • The NVIDIA RTX 6000 Ada can be scaled to fit 4x in a Workstation or up to 8x in a 4U Server.
2. Multi-GPU Scalability:
   • Large-scale deployments see up to 25x performance gains with NVIDIA H100 GPUs, making them indispensable for high-throughput CFD projects.
   • These speedups allow organizations to run multiple simulations in parallel, optimizing designs faster than ever before.

Industries ranging from aerospace to automotive can benefit from incorporating GPU-accelerated computing to their infrastructure, reducing time-to-market while improving product quality. The ability to simulate larger, more detailed models also opens doors for innovation in complex systems and even be able to simulate models previously impossible.

GPUs are not cheap but they are drastically worth time saving investment. Time is money. The ability to not only run subsequent simulations but run them fast dramatically improves business workflow and quality. A multi-node CPU cluster is costly to run and maintain versus a single GPU-accelerated system.

If you're considering upgrading your hardware for CFD workflows, leveraging GPUs is no longer an option—it's a necessity. If your workflow can take advantage of GPU acceleration, invest in your computing as more and more solvers continue to migrate and adopt GPUs. Contact us to learn how Exxact can tailor GPU-powered systems to meet your CFD needs. Thanks to ATA Engineering for running these benchmarks on Exxact hardware.