chore: skip Iceberg and Spark SQL test workflows on test-only changes#4023
Open
andygrove wants to merge 2 commits intoapache:mainfrom
Open
chore: skip Iceberg and Spark SQL test workflows on test-only changes#4023andygrove wants to merge 2 commits intoapache:mainfrom
andygrove wants to merge 2 commits intoapache:mainfrom
Conversation
Extend paths-ignore in iceberg_spark_test.yml and spark_sql_test.yml so PRs that touch only Comet's own test code or resources do not trigger these external-project test workflows. Adds spark/src/test/**, common/src/test/**, fuzz-testing/**, spark-integration/**, and dev/** to the ignore list. The broader spark/src/test/** pattern subsumes the previous benchmark-only entry.
The dev/ directory contains Spark source diffs applied to the external Spark checkout, so changes there can affect Spark SQL test behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A. Small CI chore prompted by #4021, which only added a SQL test resource file but still triggered the Iceberg and Spark SQL test workflows.
Rationale for this change
The Iceberg and Spark SQL test workflows run the Iceberg and Spark SQL test suites against Comet as a plugin. Their behavior depends on Comet plugin code (native, common/main, spark/main), not on Comet's own test code or standalone subprojects. Today these workflows run even when a PR touches only Comet's own tests, which wastes CI time and noise.
#4021 is a concrete example: it added a single file under
spark/src/test/resources/sql-tests/...and still fired both workflows.What changes are included in this PR?
Extend
paths-ignoreon thepushandpull_requesttriggers oficeberg_spark_test.ymlandspark_sql_test.ymlto also skip:spark/src/test/**(Comet's own test code and SQL test resources)common/src/test/**(Comet's own common-module tests)fuzz-testing/**(standalone fuzz-testing project)spark-integration/**(standalone integration-test project)The broader
spark/src/test/**pattern subsumes the previousspark/src/test/scala/org/apache/spark/sql/benchmark/**entry, which was removed.Note:
dev/**is intentionally NOT added, becausedev/diffs/contains Spark source patches that are applied to the external Spark checkout at test time and therefore do affect external-project test behavior.pr_build_linux.ymlandpr_build_macos.ymlare unchanged: they run Comet's own tests, so test-only changes must still trigger them. Mixed PRs (any src change alongside test changes) still trigger the Iceberg and Spark SQL workflows, because GitHub only skips when every changed file matches the ignore list.How are these changes tested?
CI configuration change. The new triggers will be exercised by subsequent PRs. The YAML files were validated and the diff is limited to additions in
paths-ignoreplus the single redundant-entry removal noted above.