feat: support TRIGGER ON UPDATE and SCHEDULE EVERY for MV and ST - #1434
Merged
Conversation
Adds the full Databricks refresh-schedule grammar on materialized views and streaming tables: SCHEDULE CRON, SCHEDULE EVERY, and TRIGGER ON UPDATE [AT MOST EVERY INTERVAL ...]. - RefreshConfig is a discriminated config with parse-time validation on the new modes. - Idempotent runs no longer emit spurious ALTERs across user/server unit differences. - Auto-REFRESH is suppressed for every/on_update modes. Coverage: - Unit tests for parser, validator, diff, and refresh-schedule macros. - Functional round-trip tests per mode for both MV and ST. - Functional lifecycle tests walking each relation through every mode transition plus a non-refresh-component change.
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||
Drop log-line and SQL-emission assertions, keep only mode-value checks on the resulting relation. Remove tests that only inspected dbt logs.
This comment was marked as duplicate.
This comment was marked as duplicate.
1 similar comment
Contributor
|
/integration-test |
|
Integration tests dispatched for PR #1434 by @tejassp-db. Track progress in the Actions tab. |
|
Integration results for PR #1434 — UC cluster ❌ cancelled · SQL warehouse ❌ cancelled · All-purpose cluster ❌ failure |
Collaborator
Author
|
/integration-test |
|
Integration tests dispatched for PR #1434 by @sd-db. Track progress in the Actions tab. |
|
Integration results for PR #1434 — UC cluster ⏩ skipped · SQL warehouse ⏩ skipped · All-purpose cluster ⏩ skipped · Shard coverage ⏩ skipped |
Collaborator
Author
|
/integration-test |
|
Integration tests dispatched for PR #1434 by @sd-db. Track progress in the Actions tab. |
|
Integration results for PR #1434 — UC cluster ✅ success · SQL warehouse ✅ success · All-purpose cluster ✅ success · Shard coverage ✅ success |
tejassp-db
reviewed
May 11, 2026
tejassp-db
approved these changes
May 12, 2026
When a user sets `schedule = {'cron': '...'}` without `time_zone_value`,
the warehouse stores it as `Etc/UTC` in `DESCRIBE EXTENDED`. Previously
`RefreshConfig.__eq__` compared `"UTC" != "ETC/UTC"`, so dbt emitted a
no-op `ALTER` on every subsequent run.
Adds `_canonical_tz()` mapping None / UTC / Etc/UTC to the same value;
`__eq__` and `__hash__` use it on the CRON branch.
Test coverage:
- New EVERY-input matrix test per file (MV + ST) walks one relation
through HOUR / DAY / WEEK via `on_configuration_change: apply`,
asserting each round-trips against the warehouse.
- The existing CRON lifecycle step now uses a no-TZ fixture so the
`Etc/UTC` round-trip is exercised on the existing CRON path.
- Unit tests cover the equivalence + parametrized server-emitted shapes
(TRIGGER intervals normalized to SECOND, EVERY plural forms).
…trigger-schedule-syntax # Conflicts: # tests/functional/adapter/materialized_view_tests/fixtures.py
The schedule-mode test files inlined six SQL constants each
(ST_EVERY_2_HOURS, ST_ON_UPDATE_BARE, ST_CRON, etc.) — out of line with
the convention used elsewhere in `streaming_tables/fixtures.py` and
`materialized_view_tests/fixtures.py` (`streaming_table`,
`complex_streaming_table`, `materialized_view`).
Moved each to its respective fixtures.py with snake_case names and
deduped two cases:
- `ST_NO_SCHEDULE` is now the pre-existing `fixtures.streaming_table`.
- `ST_EVERY_2_HOURS` / `MV_EVERY_2_HOURS` are now defined via the
`streaming_table_with_every("2 HOURS")` / `materialized_view_with_every`
helper added in the previous commit.
- Added `materialized_view_no_schedule` (the existing `materialized_view`
fixture has cron + tblproperties, so it can't double as the no-schedule
case).
No behavioral change to the tests.
`test_every_mode_roundtrip` only asserted `refresh.mode.value == "every"`
for `EVERY 2 HOURS`. The `TestStreamingTableEveryAcceptedInputs` matrix
test (added previously) now walks HOUR / DAY / WEEK with strict equality
+ no-diff checks — strictly stronger coverage.
Remove the redundant test and the `streaming_table_every_2_hours` /
`materialized_view_every_2_hours` fixtures; the few remaining call sites
in the lifecycle and drop-and-readd tests use the
`streaming_table_with_every("2 HOURS")` / `materialized_view_with_every`
helper inline.
Both removed cases assert exactly what the lifecycle test already does: - `TestStreamingTableManualMode` / `TestMaterializedViewManualMode` — identical to lifecycle's first step (CREATE no-schedule → assert manual). - `test_on_update_rate_limited_mode_roundtrip` — same `"900" in refresh.at_most_every` assertion as the lifecycle ON_UPDATE step. Leaves `test_on_update_bare_mode_roundtrip` (only coverage for bare TRIGGER ON UPDATE on fresh CREATE; lifecycle doesn't include a bare on_update transition).
This was referenced May 15, 2026
sd-db
added a commit
that referenced
this pull request
May 15, 2026
…R 15.4 (#1461) ### Description PR #1434 added `from pydantic import model_validator` to `refresh.py`, a v2-only symbol. This breaks adapter import on Databricks Jobs DBR 15.4 LTS, which ships pydantic v1.10.x. This PR rewrites the three v2-only call sites to v1-compatible equivalents that also work in v2 via the deprecation shim: | Before (v2 only) | After (v1 + v2) | |---|---| | `@model_validator(mode="after")` | `@root_validator(skip_on_failure=True)` | | `self.model_construct(**self.model_dump(), ...)` | `self.copy(update=...)` | ### Verification - Unit tests: 1081 passed, 6 skipped - Local smoke (pydantic v2): 56 PASS / 0 ERROR - Databricks Jobs verify on DBR 15.4 LTS (pydantic v1.10.x) - Reproduced the post-merge `Min-Deps Parse` workflow (#1460) locally against this branch: `uv pip install --resolution lowest-direct .` resolved pydantic to 1.10.0; `dbt parse --project-dir tests/min_deps_smoke --profiles-dir tests/min_deps_smoke` succeeds, confirming the fix clears the same import path that the post-merge gate exercises. ### Checklist - [x] Added changes to CHANGELOG.md - [x] Unit tests pass - [x] Functional/smoke tests pass on Databricks Jobs (DBR 15.4 LTS)
mirnawong1
pushed a commit
to dbt-labs/docs.getdbt.com
that referenced
this pull request
Jun 12, 2026
… MVs and streaming tables (databricks/dbt-databricks#1434) Materialized views and streaming tables now support three refresh-schedule modes via the `schedule` config: cron (`schedule: { cron, time_zone_value }`), interval (`schedule: { every: '2 HOURS' }` -> SCHEDULE EVERY, units HOURS/DAYS/WEEKS), and on-update trigger (`schedule: { on_update: true, at_most_every: '15 MINUTES' }` -> TRIGGER ON UPDATE [AT MOST EVERY INTERVAL ...], minimum 60 seconds). Exactly one of cron/every/on_update may be set, and time_zone_value is only valid alongside cron. For the interval and on-update modes, dbt no longer issues a manual REFRESH on no-op re-runs since Databricks auto-manages the refresh.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the full Databricks refresh-schedule grammar on materialized views and streaming tables: `SCHEDULE CRON`, `SCHEDULE EVERY`, and `TRIGGER ON UPDATE [AT MOST EVERY INTERVAL ...]`.
Closes #1293.
User-facing config:
Test plan