Skip to content

feat: improve common subplan elimination#19769

Merged
SkyFan2002 merged 6 commits intodatabendlabs:mainfrom
SkyFan2002:subplan_1
Apr 29, 2026
Merged

feat: improve common subplan elimination#19769
SkyFan2002 merged 6 commits intodatabendlabs:mainfrom
SkyFan2002:subplan_1

Conversation

@SkyFan2002
Copy link
Copy Markdown
Member

@SkyFan2002 SkyFan2002 commented Apr 23, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Q88 Performance

Measured with the same TPC-DS Q88 query using bendsql, comparing enable_cse_optimizer=0 and enable_cse_optimizer=1. The result uses 1 warmup run and 4 measured runs, and reports server-side execution time.

Setting Avg Min Max
enable_cse_optimizer=0 54.8s 54.6s 55.1s
enable_cse_optimizer=1 24.5s 24.4s 24.6s

This improves Q88 by about 55%, reducing average runtime from 54.8s to 24.5s.

Key Changes

  • Run common subplan elimination before recursive rewrite so reusable CTE plans are formed earlier.
  • Preserve join predicates during CTE filter pushdown, deduplicate pushed filters, and reuse pulled-up cross-join shapes.
  • Sync MaterializedCTERef statistics with producer estimates so EXPLAIN reflects the updated CTE plan accurately.
  • Add regression coverage for the simple CSE case and the TPC-DS Q88 explain plan.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions Bot added the pr-feature this PR introduces a new feature to the codebase label Apr 23, 2026
@SkyFan2002 SkyFan2002 added the ci-benchmark Benchmark: run all test label Apr 23, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Docker Image for PR

  • tag: pr-19769-c7eb475-1776943208

note: this image tag is only available for internal use.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 461194d7e3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Keep cross join operand order in the CSE signature so
side-swapped plans are not deduplicated and remapped
positionally.

Add a regression test for swapped cross join operands and
refresh the affected explain baseline for cte_prune_columns.
@SkyFan2002
Copy link
Copy Markdown
Member Author

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 337dd2c782

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@databendlabs databendlabs deleted a comment from github-actions Bot Apr 24, 2026
Comment thread src/query/sql/src/planner/optimizer/optimizers/cse/analyze.rs
SkyFan2002 and others added 2 commits April 28, 2026 10:27
Add recursive stack protection to refresh_scan_ids when rebuilding CSE materialized definitions.

Addresses review comment #3150952123.
@SkyFan2002 SkyFan2002 added this pull request to the merge queue Apr 29, 2026
Merged via the queue into databendlabs:main with commit f792cd3 Apr 29, 2026
91 checks passed
@SkyFan2002 SkyFan2002 deleted the subplan_1 branch April 29, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-benchmark Benchmark: run all test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants