The accelerating outflow of ice in Antarctica or Greenland due to a warming climate or the geodynamic processes shaping the Earth share common computational challenges: extreme-scale high-performance computing (HPC) which requires the next-generation of numerical models, parallel solvers and supercomputers. We here present a fresh approach to modern HPC and share our experience running Julia on thousands of graphical processing units (GPUs).

We present an efficient approach for writing architecture-agnostic parallel high-performance stencil computations in Julia. Powerful metaprogramming, costless abstractions and multiple dispatch enable writing a single code that is usable for both productive prototyping on a single CPU and for production runs on GPU or CPU workstations or supercomputers. Performance similar to CUDA C is achievable, which is typically a large improvement over reachable performance with CUDA.jl Array programming.

We present a straightforward approach for distributed parallelization of stencil-based Julia applications on a regular staggered grid using GPUs and CPUs. The approach allows to leverage remote direct memory access and was shown to enable close to ideal weak scaling of real-world applications on thousands of GPUs. The communication performed can be easily hidden behind computation.