In this post we provide answers to the most common questions our EDEM users ask us regarding GPU and multi-GPU. Find out more about the EDEM GPU solver engine, why and when to use GPU and multi-GPU, recommended cards and what speed-up to expect.
1. What is EDEM GPU and why use it?
The EDEM GPU solver engine allows users to run EDEM simulations using their computers Graphical Processing Unit (GPU). Unlike traditional desktop CPUs, GPUs contain many thousands of compute cores and the Discrete Element Method (DEM) is particularly suited to scaling across these high number of cores. This means the EDEM GPU solver can run simulations faster and also that users can now run larger simulations than what was possible on CPU alone.
2. What framework is EDEM GPU using?
EDEM GPU is written using OpenCL in order to offer support across operating systems (Windows and Linux) and to ensure that a wide range of GPU hardware is supported. This means users can choose to run on a range of hardware from the main GPU manufacturers, AMD and NVIDIA.
3. Is EDEM GPU single or double precision?
EDEM GPU is fully double precision. This is particularly important for computational accuracy of DEM simulations where minimizing errors from floating point arithmetic will increase the accuracy of the overall solution.
4. What cards do you recommend?
We recommend that users choose workstation or data-center grade GPUs for running EDEM simulations.
At the minimum you should look at the following:
NVIDIA GeForce GTX 1080 Ti
NVIDIA GeForce RTX 2080 Ti
NVIDIA TITAN series
NVIDIA Quadro P6000
AMD Radeon Pro WX7100
Our recommended GPUs include the following:
NVIDIA Quadro GV100
NVIDIA Tesla V100
NVIDIA Quadro GP100
AMD Radeon Pro WX9100
The range of GPU cards available and the technology behind them is constantly changing and moving fast. Our list of minimum spec cards typically retail for ~USD $1000 and our list of recommended cards go for up to ~USD $11,000 with key differentiators typically being number of cores, size of available memory and FP64 (double precision) performance.
Customers looking to purchase a GPU to run EDEM simulations should check our GPU Benchmarking document which also highlights the specification of cards which have been tested with the latest version of the EDEM GPU solver. In general, cards with a large physical memory and high FP64 performance perform well when running large EDEM simulations.
If you’re not sure of what hardware you need please get in touch and our engineers will be happy to talk about results from our most recent rounds of testing.
5. Where is the GPU most effective?
In most situations, running simulations on GPU becomes more effective than running on CPU above particle counts of 10k and you start to see significant speed up above 100k. In benchmark simulations containing 1 million particles, simulations run on the Nvidia Quadro GV100 have shown speed ups of up to 12x when compared against running the same simulation on 12 x CPU. Large simulations that previously would take potentially weeks to run on CPU can now be run in under 24 hours because of the size of the speed up seen between CPU and GPU.
6. Do you have examples showing what kind of speed-up can be achieved?
Below are 3 test cases, angle of repose, hopper discharge and powder mixer – each including 1 million particles. The graph shows the speed up for various hardware – comparing 12 cores, 24 cores and 4 different GPU cards. More examples and benchmarks data are available to EDEM users on the EDEM website.
Speed up (vs 12x CPU)
7. When to use multi-GPU?
You might consider using EDEM Multi GPU when you need to speed up a large simulation (speed ups of ~30% – 40% have been seen when going from 1x to 2x GPU on simulations containing 5 to 10 million particles) but the main benefit of EDEM Multi GPU is when you need to run simulations with a very high number of particles (10’s of millions of particles). This is because beyond a certain threshold a single GPU -even with the best hardware currently available- will not have enough physical memory to run a simulation.
Memory used by the EDEM GPU solver is dependent on many factors including number of particles, number of contact and size of the simulator grid to name a few. In most cases investing in a more capable single GPU should be considered first before upgrading to multi GPU functionality if the size of your simulation requires it.
8. Can I run custom physics models (using the EDEM Application Programming Interface) on GPU?
EDEM GPU API, introduced in EDEM 2019, means you can now write and run API models directly on GPU. These models are written in OpenCL. Example GPU API models can be accessed on the EDEM User Forum.
9. How can I get access to EDEM GPU?
The EDEM GPU solver engine is directly integrated into the EDEM environment and activated by an add-on license token. Users can contact their Account manager to get started.
10. How do I go about using the GPU solver in my simulation?
Setting and running simulations with the EDEM GPU engine is as simple as enabling the GPU engine and then running EDEM the same way you would any other simulation. To get a step-by-step guide check our eLearning course: Introduction to EDEM 2019 – lessons 14.5 and 14.6 which explain how to use the GPU and multi-GPU solver engines.