Skip to content

GPU-offload capabilities for offline Ecland driver - #56

Draft
mlange05 wants to merge 34 commits into
ecmwf-ifs:mainfrom
mlange05:naml-ecland-gpu-dev
Draft

GPU-offload capabilities for offline Ecland driver#56
mlange05 wants to merge 34 commits into
ecmwf-ifs:mainfrom
mlange05:naml-ecland-gpu-dev

Conversation

@mlange05

Copy link
Copy Markdown
Contributor

Description

Note: I am filing this PR as a draft early to let reviewers assess the complexity of the PR and test the build-hpc CI pipeline. As the diff is quite large I'd be happy to break this contribution into multiple independent PRs to ease review and test assessment. Please let me know what you prefer. _

This PR brings GPU-support for the standalone Ecland offline driver for Nvidia GPUs by adding a dedicated build mode that uses the Loki source-to-source translation toolchain to auto-generate GPU-versions of the four main kernels in the offline driver. For this it substantially refactors the offline driver control flow and adjust some model-level sources.

Importantly, this does not yet enable full compatibility with the IFS-driven Ecland GPU-variant (needed for full extraction), due to the way the EC-physics package calls Ecland. However, this change constitutes a significant step in that direction and will allow further development and testing once the basic feature is established.

In more detail:

  • Add the infrastructure (Loki, OpenACC compilation) to buildsystem and CMake setup, including local Loki configuration
  • Remove CALLPAR1S and replace single parallel region with four parallel regions around SURFBC, SURFRAD, VDFMAINS1S and SURFTSTP.
  • Implement each parallel regions in a standalone OpenMP layer inside a <DRIVER>_LAYER_OPENMP subroutine
  • Add dispatch routines <DRIVER>_LAYER_PARALLEL that can call OpenMP or Loki+OpenACC GPU variants of the individual parallel region drivers. The behaviour is toggled at runtime with a namelist flag
  • Adjustments to some kernels to enable Loki processing; this includes moving large constant arrays out of compute regions and making argument array dimensions explicit.
  • Adjusting the ifsbench-based test configurations to include a shared "tech" setup, and adding launcher flags, which allow dispatching the same tests on GPU.
  • Adding a basic CI test setup for GPU-enabled tests based on the build-hpc workflow

Contributor Declaration

By opening this pull request, I affirm the following:

  • All authors agree to the Contributor License Agreement.
  • The code follows the project's coding standards.
  • I have performed self-review and added comments where needed.
  • I have added or updated tests to verify that my changes are effective and functional.
  • I have run all existing tests and confirmed they pass.

mlange05 and others added 30 commits July 17, 2026 04:31
Duplicate the current CPG1S block-loop driver as CPG1S_PARALLEL and
wire STEPO1S to use it, preserving the existing orchestration path for
comparison while CPG1S is prepared for layer-based refactoring.
This introduces the _LAYER_PARALLEL wrapper routines and creates
dedicated OpenMP-parallel wrappers for SURFBC, SURFRAD, SURFTSTP
and VDFMAIN1S to mimic the corresponding layout in EC_PHYS.
…host

For validation purposes, we keep these on host for now, but use the
SGET_ F-API pointer semantics.
This uses `!$acc declare` in the defining model and forces an update
after the initial setup step.
This is required to offload the YUSRF object correctly in GPU code
paths.
Imported from IFS commit fbb5e13e5b79e53b076e10dc04a5a7b514edb3dd with paths adapted for ecland.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants