Skip to content

Run smoke matrix on push, full matrix on demand #14432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
marawan206 wants to merge 1 commit into
base: master
Choose a base branch
from
+51 −59
Open

Run smoke matrix on push, full matrix on demand #14432

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 51 additions & 59 deletions .github/workflows/test-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# This is the GitHub Workflow that drives automatic full-GPU-enabled tests of all new commits to the master branch of ComfyUI
# This is the GitHub Workflow that drives full-GPU-enabled tests of ComfyUI.
# Results are reported as checkmarks on the commits, as well as onto https://ci.comfy.org/
#
# Trigger policy:
# push to master/release -> a lightweight "smoke" run (one stable config) for a fast per-commit signal
# workflow_dispatch -> operator-selected scope; "full" runs the complete supported matrix on demand
name: Full Comfy CI Workflow Runs
on:
push:
Expand All @@ -15,85 +19,73 @@ on:
- '.github/**'
- 'web/**'
workflow_dispatch:
inputs:
scope:
description: "Test scope: 'smoke' = one stable config, 'full' = all supported Python versions + nightly"
type: choice
options:
- smoke
- full
default: full

jobs:
# Resolve the test scope from the trigger:
# push -> smoke (cheap per-commit signal on master)
# workflow_dispatch -> the scope chosen by the operator (defaults to full)
# Expanding coverage later (new Python versions, etc.) is a one-line edit to the JSON below.
prepare:
runs-on: ubuntu-latest
outputs:
stable_python: ${{ steps.scope.outputs.stable_python }}
run_nightly: ${{ steps.scope.outputs.run_nightly }}
Comment on lines 31 to +40

@coderabbitai coderabbitai Bot Jun 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add explicit least-privilege permissions for this workflow/jobs.

The new prepare + refactored test jobs run with default token permissions because no permissions block is declared. This weakens CI hardening and is flagged in the changed regions. Please set explicit minimal permissions (workflow-level and/or per-job).

Suggested hardening patch 
 name: Full Comfy CI Workflow Runs
 
+permissions:
+  contents: read
+
 on:
   push:
     branches:
       - master
       - release/**

Also applies to: 56-57, 76-77

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test-ci.yml around lines 31 - 40, The workflow currently
relies on default GITHUB_TOKEN permissions; add an explicit least-privilege
permissions block to harden the workflow and the affected jobs (notably the
prepare job that exposes outputs stable_python and run_nightly and the other
test jobs referenced). For example, add a top-level permissions section (or
per-job permissions for prepare and the test jobs) granting only the minimal
scopes needed (e.g., contents: read, actions: read, checks: write if tests
report statuses) and remove/override default broad permissions; ensure any job
that needs extra rights explicitly requests them rather than inheriting
defaults.

Source: Linters/SAST tools

steps:
- name: Resolve scope
id: scope
shell: bash
run: |
SCOPE="${{ github.event_name == 'workflow_dispatch' && inputs.scope || 'smoke' }}"
echo "Trigger=${{ github.event_name }} resolved scope=$SCOPE"
if [ "$SCOPE" = "full" ]; then
echo 'stable_python=["3.10", "3.11", "3.12"]' >> "$GITHUB_OUTPUT"
echo 'run_nightly=true' >> "$GITHUB_OUTPUT"
else
echo 'stable_python=["3.12"]' >> "$GITHUB_OUTPUT"
echo 'run_nightly=false' >> "$GITHUB_OUTPUT"
fi

test-stable:
needs: prepare
strategy:
fail-fast: false
matrix:
# os: [macos, linux, windows]
# os: [macos, linux]
os: [linux]
python_version: ["3.10", "3.11", "3.12"]
cuda_version: ["12.1"]
torch_version: ["stable"]
include:
# - os: macos
# runner_label: [self-hosted, macOS]
# flags: "--use-pytorch-cross-attention"
- os: linux
runner_label: [self-hosted, Linux]
flags: ""
# - os: windows
# runner_label: [self-hosted, Windows]
# flags: ""
runs-on: ${{ matrix.runner_label }}
# os: [macos, linux, windows] # mac/windows self-hosted runners currently disabled
python_version: ${{ fromJSON(needs.prepare.outputs.stable_python) }}
# CUDA is the comfy-action default (12.1); bump alongside the matrix-expansion PR.
runs-on: [self-hosted, Linux]
steps:
- name: Test Workflows
uses: comfy-org/comfy-action@main
with:
os: ${{ matrix.os }}
os: linux
python_version: ${{ matrix.python_version }}
torch_version: ${{ matrix.torch_version }}
torch_version: stable
google_credentials: ${{ secrets.GCS_SERVICE_ACCOUNT_JSON }}
comfyui_flags: ${{ matrix.flags }}

# test-win-nightly:
# strategy:
# fail-fast: true
# matrix:
# os: [windows]
# python_version: ["3.9", "3.10", "3.11", "3.12"]
# cuda_version: ["12.1"]
# torch_version: ["nightly"]
# include:
# - os: windows
# runner_label: [self-hosted, Windows]
# flags: ""
# runs-on: ${{ matrix.runner_label }}
# steps:
# - name: Test Workflows
# uses: comfy-org/comfy-action@main
# with:
# os: ${{ matrix.os }}
# python_version: ${{ matrix.python_version }}
# torch_version: ${{ matrix.torch_version }}
# google_credentials: ${{ secrets.GCS_SERVICE_ACCOUNT_JSON }}
# comfyui_flags: ${{ matrix.flags }}
comfyui_flags: ""

test-unix-nightly:
needs: prepare
if: ${{ needs.prepare.outputs.run_nightly == 'true' }}
strategy:
fail-fast: false
matrix:
# os: [macos, linux]
os: [linux]
python_version: ["3.11"]
cuda_version: ["12.1"]
torch_version: ["nightly"]
include:
# - os: macos
# runner_label: [self-hosted, macOS]
# flags: "--use-pytorch-cross-attention"
- os: linux
runner_label: [self-hosted, Linux]
flags: ""
runs-on: ${{ matrix.runner_label }}
runs-on: [self-hosted, Linux]
steps:
- name: Test Workflows
uses: comfy-org/comfy-action@main
with:
os: ${{ matrix.os }}
os: linux
python_version: ${{ matrix.python_version }}
torch_version: ${{ matrix.torch_version }}
torch_version: nightly
google_credentials: ${{ secrets.GCS_SERVICE_ACCOUNT_JSON }}
comfyui_flags: ${{ matrix.flags }}
comfyui_flags: ""
Loading