Skip to content

Move all Python test dependencies to pyproject#4256

Open
SandSnip3r wants to merge 1 commit into
pytorch:mainfrom
SandSnip3r:dev-vstone-update-ci-deps
Open

Move all Python test dependencies to pyproject#4256
SandSnip3r wants to merge 1 commit into
pytorch:mainfrom
SandSnip3r:dev-vstone-update-ci-deps

Conversation

@SandSnip3r
Copy link
Copy Markdown
Collaborator

@SandSnip3r SandSnip3r commented May 12, 2026

Install Python test dependencies from pyproject dependency groups in CI to avoid duplicated requirements.

Description

Summary

This PR ports Python test dependency installation in CI from tests/py/requirements.txt to pyproject.toml dependency groups, so local development and CI use the same dependency source of truth.

Problem

The test dependencies were defined in more than one place and had drifted. In particular, tests/py/requirements.txt pinned transformers==4.53.1, while pyproject.toml specified transformers>=5.0.0.

That was risky because GitHub CI was installing from tests/py/requirements.txt, while local development with uv uses the dependency groups in pyproject.toml. As a result, CI and local dev could resolve meaningfully different test environments.

What Changed

  • Moved the shared Python test dependencies into the test, test-ext, and quantization dependency groups in pyproject.toml.

  • Updated .github/scripts/install-torch-tensorrt.sh to install test dependencies with:

    pip install --group test --group test-ext --group quantization
    
  • Updated noxfile.py to install the same dependency groups instead of tests/py/requirements.txt.

  • Reduced tests/py/requirements.txt to a compatibility stub for tooling that still expects the file to exist.

  • Updated the release checklist to point maintainers at pyproject.toml dependency groups instead of the old test requirements file.

Design Decisions

The main design choice was to make pyproject.toml the source of truth for shared test dependencies. This matches the local uv workflow and avoids maintaining duplicate package constraints in both dependency groups and a requirements file.

CI now uses native pip dependency group support instead of a custom parser or generated requirements file. This keeps the CI path simple and close to the intended interface for PEP 735 dependency groups.

install-torch-tensorrt.sh upgrades to pip>=25.1 before installing groups because pip --group support requires a recent pip. The script also installs tomli only for Python versions before 3.11 because it parses pyproject.toml to recover the torchvision requirement used later in the script.

The old tests/py/requirements.txt file is intentionally left in place as an empty compatibility stub because there is still non-CI tooling that references it by path. It no longer contains dependency constraints, so it cannot drift from pyproject.toml.

Type of change

  • CI Refactor

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@meta-cla meta-cla Bot added the cla signed label May 12, 2026
@SandSnip3r SandSnip3r requested a review from lanluo-nvidia May 12, 2026 20:33
@github-actions github-actions Bot added documentation Improvements or additions to documentation component: tests Issues re: Tests component: build system Issues re: Build system labels May 12, 2026
Copy link
Copy Markdown
Collaborator

@lanluo-nvidia lanluo-nvidia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
noxfile.py is deprecated anyway, we no longer use nox for pytest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: build system Issues re: Build system component: tests Issues re: Tests documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants