Skip to content

Commit a80a3a8

Browse files
authored
chore: promote array_insert to compatible and consolidate expression docs (#4065)
1 parent 499f24d commit a80a3a8

7 files changed

Lines changed: 13 additions & 42 deletions

File tree

.claude/skills/audit-comet-expression/SKILL.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -311,17 +311,16 @@ After implementing tests, tell the user how to run them:
311311

312312
---
313313

314-
## Step 8: Update the Expression Audit Log
314+
## Step 8: Update the Expression Support Doc
315315

316-
After completing the audit (whether or not tests were added), append a row to the audit log at
317-
`docs/source/contributor-guide/expression-audit-log.md`.
316+
After completing the audit (whether or not tests were added), add sub-bullets under the expression's
317+
entry in `docs/source/contributor-guide/spark_expressions_support.md`.
318318

319-
The row should include:
319+
Add one sub-bullet per Spark version checked, each including:
320320

321-
- Expression name
322-
- Spark versions checked (e.g. 3.4.3, 3.5.8, 4.0.1)
321+
- Spark version (e.g. 3.4.3, 3.5.8, 4.0.1)
323322
- Today's date
324-
- A brief summary of findings (behavioral differences, bugs found/fixed, tests added, known incompatibilities)
323+
- A brief note for any version-specific finding (behavioral difference, known incompatibility); omit if nothing notable
325324

326325
---
327326

docs/source/contributor-guide/adding_a_new_expression.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Before you start, have a look through [these slides](https://docs.google.com/pre
2525

2626
## Finding an Expression to Add
2727

28-
You may have a specific expression in mind that you'd like to add, but if not, you can review the [expression coverage document](https://github.com/apache/datafusion-comet/blob/main/docs/spark_expressions_support.md) to see which expressions are not yet supported.
28+
You may have a specific expression in mind that you'd like to add, but if not, you can review the [expression coverage document](spark_expressions_support.md) to see which expressions are not yet supported.
2929

3030
## Implementing the Expression
3131

docs/source/contributor-guide/expression-audit-log.md

Lines changed: 0 additions & 32 deletions
This file was deleted.

docs/source/contributor-guide/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ Debugging Guide <debugging>
3333
Benchmarking Guide <benchmarking>
3434
Adding a New Operator <adding_a_new_operator>
3535
Adding a New Expression <adding_a_new_expression>
36+
Supported Spark Expressions <spark_expressions_support>
3637
Tracing <tracing>
3738
Profiling <profiling>
3839
Spark SQL Tests <spark-sql-tests.md>

docs/spark_expressions_support.md renamed to docs/source/contributor-guide/spark_expressions_support.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,9 @@
8989
- [x] array_distinct
9090
- [x] array_except
9191
- [x] array_insert
92+
- Spark 3.4.3 audited 2026-04-02
93+
- Spark 3.5.8 audited 2026-04-02
94+
- Spark 4.0.1 audited 2026-04-02 (pos=0 error message differs from Spark)
9295
- [x] array_intersect
9396
- [x] array_join
9497
- [x] array_max

docs/source/user-guide/latest/expressions.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -245,7 +245,7 @@ Comet supports using the following aggregate functions within window contexts wi
245245
| ArrayDistinct | Yes | |
246246
| ArrayExcept | No | |
247247
| ArrayFilter | Yes | Only supports case where function is `IsNotNull` |
248-
| ArrayInsert | No | |
248+
| ArrayInsert | Yes | |
249249
| ArrayIntersect | No | |
250250
| ArrayJoin | No | |
251251
| ArrayMax | Yes | |

spark/src/main/scala/org/apache/comet/serde/arrays.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -390,7 +390,7 @@ object CometArrayJoin extends CometExpressionSerde[ArrayJoin] {
390390

391391
object CometArrayInsert extends CometExpressionSerde[ArrayInsert] {
392392

393-
override def getSupportLevel(expr: ArrayInsert): SupportLevel = Incompatible(None)
393+
override def getSupportLevel(expr: ArrayInsert): SupportLevel = Compatible()
394394

395395
override def convert(
396396
expr: ArrayInsert,

0 commit comments

Comments
 (0)