Skip to content

Commit 03530e9

Browse files
committed
Update docs.
1 parent 71327c1 commit 03530e9

2 files changed

Lines changed: 6 additions & 6 deletions

File tree

docs/source/contributor-guide/parquet_scans.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ The following features are not supported by either scan implementation, and Come
4646
- Spark's Datasource V2 API. When `spark.sql.sources.useV1SourceList` does not include `parquet`, Spark uses the
4747
V2 API for Parquet scans. The DataFusion-based implementations only support the V1 API.
4848
- Spark metadata columns (e.g., `_metadata.file_path`)
49-
- No support for Dynamic Partition Pruning (DPP)
49+
- No support for AQE Dynamic Partition Pruning (DPP). Non-AQE DPP is supported.
5050

5151
The following shared limitation may produce incorrect results without falling back to Spark:
5252

docs/source/contributor-guide/roadmap.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -42,14 +42,14 @@ more Spark SQL tests and fully implementing ANSI support ([#313]) for all suppor
4242

4343
### Dynamic Partition Pruning
4444

45-
Iceberg table scans support Dynamic Partition Pruning (DPP) filters generated by Spark's `PlanDynamicPruningFilters`
46-
optimizer rule ([#3349]). However, we still need to bring this functionality to our Parquet reader. Furthermore,
47-
Spark's `PlanAdaptiveDynamicPruningFilters` optimizer rule runs after Comet's rules, so DPP with Adaptive Query
48-
Execution requires a redesign of Comet's plan translation. We are focused on implementing DPP to keep Comet competitive
49-
with benchmarks that benefit from this feature like TPC-DS. This effort can be tracked at [#3510].
45+
Both Iceberg table scans and Parquet V1 native scans (`CometNativeScanExec`) support non-AQE Dynamic Partition Pruning
46+
(DPP) filters generated by Spark's `PlanDynamicPruningFilters` optimizer rule ([#3349], [#3511]). However, Spark's
47+
`PlanAdaptiveDynamicPruningFilters` optimizer rule runs after Comet's rules, so DPP with Adaptive Query Execution
48+
requires a redesign of Comet's plan translation. This effort can be tracked at [#3510].
5049

5150
[#3349]: https://github.com/apache/datafusion-comet/pull/3349
5251
[#3510]: https://github.com/apache/datafusion-comet/issues/3510
52+
[#3511]: https://github.com/apache/datafusion-comet/pull/3511
5353

5454
## Ongoing Improvements
5555

0 commit comments

Comments
 (0)