refactor(ci): consolidate yq and bicep installs into make targets#12248
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned Files
|
There was a problem hiding this comment.
Pull request overview
This PR centralizes installation of CI tooling (notably yq and the Bicep CLI) behind new Make targets and hardened installer scripts, replacing duplicated inline download logic across multiple GitHub workflows and build targets.
Changes:
- Add
build/tools.mkand new installer scripts to pin tool versions and (where possible) verify SHA-256 checksums. - Update build Make targets and multiple workflows to use
make install-yq/make install-bicepinstead of bespoke inline install steps. - CI cleanup: adjust workflow concurrency/skip lists, move triage-bot label config, and remove the obsolete root
CODEOWNERS.
Review notes (blocking):
build/build.mkcurrently passes"$$(BICEP_VERSION)"-style values, which becomes shell command substitution ($(...)) and will fail at runtime..github/workflows/release.yamlintroduces::set-output, which is deprecated in GitHub Actions; it should use$GITHUB_OUTPUT.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| Makefile | Includes build/tools.mk to expose shared tool install targets. |
| CODEOWNERS | Removes obsolete root-level pointer to .github/CODEOWNERS. |
| build/tools.mk | Defines pinned yq / bicep versions + checksums and provides install-* Make targets. |
| build/scripts/install-yq.sh | New cross-platform installer for yq with checksum verification. |
| build/scripts/install-bicep.sh | New cross-platform installer for Bicep CLI with optional checksum verification and cross-arch staging support. |
| build/resource-types.mk | Updates yq prerequisite message to use make install-yq. |
| build/install-bicep.sh | Narrows script responsibility to only generating bicepconfig.json. |
| build/generate.mk | Removes old YQ_VERSION pin and updates install guidance to make install-yq. |
| build/build.mk | Updates bicep container staging to use the new Bicep installer script. |
| .github/workflows/verify-resource-types.yaml | Replaces inline yq install with make install-yq. |
| .github/workflows/validate-bicep.yaml | Replaces inline bicep install with make install-bicep (installing into ~/.rad/bin). |
| .github/workflows/unit-tests.yaml | Normalizes concurrency group expression. |
| .github/workflows/triage-bot.yaml | Points label-actions config to new .github/configs/label-actions.yaml location. |
| .github/workflows/release.yaml | Switches from action-based yq usage to installing yq and invoking it directly (but uses deprecated ::set-output). |
| .github/workflows/publish-docs.yaml | Replaces inline yq install with make install-yq. |
| .github/workflows/lint.yaml | Replaces inline yq install with make install-yq. |
| .github/workflows/functional-test-noncloud.yaml | Replaces inline yq/bicep installs with Make targets. |
| .github/workflows/functional-test-cloud.yaml | Adds centralized authorize gate and replaces inline tool installs with Make targets. |
| .github/workflows/copilot-setup-steps.yml | Replaces inline yq/bicep installs with Make targets. |
| .github/workflows/contrib-update-resource-types.yaml | Replaces inline yq install with make install-yq. |
| .github/workflows/__changes.yml | Expands ignore list and adds documentation clarifying skip behavior. |
| .github/configs/label-actions.yaml | New location/name for triage-bot label-actions config (plus small YAML cleanup). |
- Removed hardcoded yq installation steps from multiple workflows and replaced them with a single `make install-yq` command. - Updated Bicep CLI installation to use `make install-bicep` in relevant workflows. - Removed deprecated environment variables related to yq and Bicep versions from workflows. - Added new scripts for installing yq and Bicep CLI, allowing for easier version management and installation. - Updated Makefile to include new tools.mk for managing tool versions and checksums. - Adjusted concurrency group format in unit-tests workflow for improved consistency. - Changed CODEOWNERS file path in triage-bot workflow. - Deleted obsolete CODEOWNERS file. Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…ully Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…e workflow Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
a3810bd to
d71c49e
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #12248 +/- ##
==========================================
- Coverage 52.88% 52.87% -0.02%
==========================================
Files 751 751
Lines 48353 48353
==========================================
- Hits 25573 25566 -7
- Misses 20383 20387 +4
- Partials 2397 2400 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🚀
One non-blocking suggestion:
Rename the bicepconfig generator. After this change, build/install-bicep.sh no longer installs the Bicep CLI — it only generates bicepconfig.json — while the new build/scripts/install-bicep.sh does the actual CLI install. Two files sharing the basename install-bicep.sh with unrelated jobs is easy to confuse. Consider renaming build/install-bicep.sh to something like build/gen-bicepconfig.sh and updating its caller in build/build.mk.
…date install process Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…ation Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
…s into make targets (#12251) ## Description Follow-up to #12248, which consolidated the `yq` and `bicep` installs into pinned, checksum-verified `make install-<tool>` targets. This PR applies the same pattern to the remaining CI tools — **kind**, **kubectl**, **dapr**, **helm**, and **oras** — replacing inline `curl`/`wget | bash` installs and the third-party `azure/setup-helm` and `oras-project/setup-oras` actions. Each tool now has: - a generic, no-sudo `build/scripts/install-<tool>.sh` that detects OS/arch, verifies the download against a SHA-256 checksum, installs into a user-owned dir, and adds it to `$GITHUB_PATH` for later steps; and - a `build/tools.mk` block that pins the version + per-platform checksums and exposes a `make install-<tool>` target. Versions pinned in this PR: `kind` v0.32.0, `kubectl` v1.36.2, `dapr` v1.18.0, `helm` v4.2.2, `oras` v1.3.2. ### Why - **Single source of truth** for tool versions in `build/tools.mk`, instead of versions scattered across workflow `env:` blocks and step inputs. - **Supply-chain hardening**: every binary is checksum-verified; the unpinned `dapr` `install.sh | bash` pipe and the `setup-helm` / `setup-oras` third-party actions are removed. - **Consistent local + CI behavior**: the same `make install-<tool>` runs on a contributor's machine and on the runner. ### Files - New: `build/scripts/install-{kind,kubectl,dapr,helm,oras}.sh` - `build/tools.mk`: new KIND / KUBECTL / DAPR / HELM / ORAS version + checksum blocks and `install-*` targets - Workflows/action switched to `make install-<tool>` (dropping the now-unused version env vars): `build.yaml`, `lint.yaml`, `unit-tests.yaml`, `copilot-setup-steps.yml`, `functional-test-cloud.yaml`, `functional-test-noncloud.yaml`, `validate-installers.yaml`, `actions/create-kind-cluster/action.yaml` ## Type of change - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional). Follow-up to #12248. ## Contributor checklist Please verify that the PR meets the following requirements, where applicable: - An overview of proposed schema changes is included in a linked GitHub issue. - [ ] Yes - [x] Not applicable - A design document is added or updated under `eng/design-notes/` in this repository, if new APIs are being introduced. - [ ] Yes - [x] Not applicable - The design document has been reviewed and approved by Radius maintainers/approvers. - [ ] Yes - [x] Not applicable - A PR for [resource-types-contrib](https://github.com/radius-project/resource-types-contrib/) is created, if resource types or recipes are affected by the changes in this PR. - [ ] Yes - [x] Not applicable - A PR for [dashboard](https://github.com/radius-project/dashboard/) is created, if the Radius Dashboard is affected by the changes in this PR. - [ ] Yes - [x] Not applicable - A PR for the [documentation repository](https://github.com/radius-project/docs) is created, if the changes in this PR affect the documentation or any user facing updates are made. - [ ] Yes - [x] Not applicable --------- Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Description
Problem
Tool installation (yq and the Bicep CLI) was duplicated across many GitHub workflows using inline, hand-written download steps. This created several recurring problems:
latest, so different jobs could run different tool versions.Value proposition
This change centralizes tool installation behind Make targets backed by a single source of truth, making CI reproducible, verifiable, and easy to maintain:
build/tools.mkas the single source of truth for tool versions and per-platform SHA-256 checksums (yqv4.53.3, Bicepv0.42.1).build/scripts/install-yq.shandbuild/scripts/install-bicep.shinstall into a user-owned bin dir (nosudo), support Linux and macOS on amd64/arm64, verify checksums, honorGITHUB_TOKENfor GitHub API calls, and fall back to the latest release when no version is pinned.make install-yq/make install-biceptargets — every workflow now calls these instead of bespoke inline steps, so versions are pinned consistently and updated in one place.build/install-bicep.shis reduced to its sole remaining responsibility (generatingbicepconfig.json); all download/install logic now lives in the singlebuild/scripts/install-bicep.sh.build/build.mkcalls the installer for cross-arch container staging.functional-test-cloud.yamlgains an explicitauthorizegate so external/fork contributions run only after the trust check and approval gate pass.__changes.ymlignore-list, normalizes the unit-tests concurrency group, relocates the triage-bot label config to.github/configs/label-actions.yaml, and removes the obsolete rootCODEOWNERS.Changed files
build/tools.mk,build/scripts/install-yq.sh,build/scripts/install-bicep.shMakefile,build/build.mk,build/generate.mk,build/resource-types.mk,build/install-bicep.sh(stripped tobicepconfig.jsongeneration)copilot-setup-steps.yml,functional-test-cloud.yaml,functional-test-noncloud.yaml,validate-bicep.yaml,contrib-update-resource-types.yaml,verify-resource-types.yaml,lint.yaml,publish-docs.yaml,release.yaml,triage-bot.yaml,unit-tests.yaml,__changes.yml.github/triage-bot/triage-bot-config.yaml→.github/configs/label-actions.yamlCODEOWNERSType of change
N/A — minor CI/build tooling refactor (issue link optional).
Contributor checklist
Please verify that the PR meets the following requirements, where applicable:
eng/design-notes/in this repository, if new APIs are being introduced.