From a58146ccc113496f06188ec1d08817ef2c0af20f Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:05:20 -0400 Subject: [PATCH 1/9] docs: clarify YAML type expressions vs signature short names --- site/docs/extensions/index.md | 10 ++++++---- tests/README.md | 2 +- text/simple_extensions_schema.yaml | 5 ++++- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index c27aa83ba..793928756 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -58,7 +58,9 @@ Here, the choice for the name `ext` is arbitrary, as long as it does not conflic ### Function Signature -A YAML file may contain one or more functions with the same name, each with one or more implementations (impls). A specific function implementation within a YAML file can be identified using a Function Signature which consists of two components: +A YAML file may contain one or more functions with the same name, each with one or more implementations (impls). Function arguments, return values, intermediate aggregate values, and type structures in the YAML file are written as [type expressions](../types/type_parsing.md), such as `string`, `binary`, or `fixedbinary<16>`. + +A specific function implementation within a YAML file can be identified using a Function Signature which consists of two components: * Function Name: the name of the function * Argument Signature: the short type names of each argument joined with underscores @@ -74,7 +76,7 @@ The resulting function signatures look like: `:_ argument-signature = short-arg-type *("_" short-arg-type) ``` -Argument types (`short_arg_type`) are encoded using the Type Short Names given below. +Argument types (`short_arg_type`) are encoded using the Type Short Names given below. These short names are only used in function signatures; the YAML function definition itself continues to use type expressions. For example, a YAML argument or return type uses `binary`, while the corresponding function signature uses `vbin`. #### Variadic Functions @@ -86,8 +88,8 @@ A function signature uniquely identifies a function implementation within a sing #### Type Short Names -| Argument Type | Signature Name | -|---------------------------------|----------------| +| YAML Type Expression | Signature Short Name | +|---------------------------------|----------------------| | Enumeration | req | | i8 | i8 | | i16 | i16 | diff --git a/tests/README.md b/tests/README.md index 4cf25f389..25834b5e9 100644 --- a/tests/README.md +++ b/tests/README.md @@ -213,7 +213,7 @@ Use short names listed in https://substrait.io/extensions/#function-signature-co - **str**: Variable-length string - **fchar**: Fixed-length string `fixedchar` - **vchar**: Variable-length string `varchar` -- **vbin**: Fixed-length binary `fixedbinary` +- **vbin**: Variable-length binary `binary` - **date**: Date - **iyear**: Interval year - **iday**: Interval days `interval_day

` diff --git a/text/simple_extensions_schema.yaml b/text/simple_extensions_schema.yaml index 199872ca1..77e1a6081 100644 --- a/text/simple_extensions_schema.yaml +++ b/text/simple_extensions_schema.yaml @@ -75,7 +75,10 @@ properties: $defs: type: - description: String values are Substrait type expressions parsed by grammar/SubstraitType.g4. + description: >- + String values are Substrait type expressions parsed by grammar/SubstraitType.g4. + Use type expression names here, such as string, binary, or fixedbinary<16>. + Do not use function signature short names such as str, vbin, or fbin. oneOf: - type: string # any data type - type: object # syntactic sugar for a non-nullable named struct From d24fe991a355e3e3a199098693b5faf1f347513c Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:08:06 -0400 Subject: [PATCH 2/9] docs: keep function signature intro focused --- site/docs/extensions/index.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index 793928756..40679f1ae 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -58,9 +58,7 @@ Here, the choice for the name `ext` is arbitrary, as long as it does not conflic ### Function Signature -A YAML file may contain one or more functions with the same name, each with one or more implementations (impls). Function arguments, return values, intermediate aggregate values, and type structures in the YAML file are written as [type expressions](../types/type_parsing.md), such as `string`, `binary`, or `fixedbinary<16>`. - -A specific function implementation within a YAML file can be identified using a Function Signature which consists of two components: +A YAML file may contain one or more functions with the same name, each with one or more implementations (impls). A specific function implementation within a YAML file can be identified using a Function Signature which consists of two components: * Function Name: the name of the function * Argument Signature: the short type names of each argument joined with underscores From 85f6c6a4fdc67dc59ab93a9f7d284906435d994c Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:09:48 -0400 Subject: [PATCH 3/9] docs: revert extension signature wording --- site/docs/extensions/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index 40679f1ae..c27aa83ba 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -74,7 +74,7 @@ The resulting function signatures look like: `:_ argument-signature = short-arg-type *("_" short-arg-type) ``` -Argument types (`short_arg_type`) are encoded using the Type Short Names given below. These short names are only used in function signatures; the YAML function definition itself continues to use type expressions. For example, a YAML argument or return type uses `binary`, while the corresponding function signature uses `vbin`. +Argument types (`short_arg_type`) are encoded using the Type Short Names given below. #### Variadic Functions @@ -86,8 +86,8 @@ A function signature uniquely identifies a function implementation within a sing #### Type Short Names -| YAML Type Expression | Signature Short Name | -|---------------------------------|----------------------| +| Argument Type | Signature Name | +|---------------------------------|----------------| | Enumeration | req | | i8 | i8 | | i16 | i16 | From 65ea1f4b898a877f9ded78c8fb2f85fb495aaf1e Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:10:07 -0400 Subject: [PATCH 4/9] docs: clarify type short name table headers --- site/docs/extensions/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index c27aa83ba..23969ac52 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -86,8 +86,8 @@ A function signature uniquely identifies a function implementation within a sing #### Type Short Names -| Argument Type | Signature Name | -|---------------------------------|----------------| +| YAML Type Expression | Signature Short Name | +|---------------------------------|----------------------| | Enumeration | req | | i8 | i8 | | i16 | i16 | From 5df46c1cc6b79bcb85ff46b25b2fb119886f5ff0 Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:13:47 -0400 Subject: [PATCH 5/9] docs: note type expressions vs signature short names --- site/docs/extensions/index.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index 23969ac52..813d18b5b 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -76,6 +76,9 @@ The resulting function signatures look like: `:_ Argument types (`short_arg_type`) are encoded using the Type Short Names given below. +!!! note "Type expressions vs. signature short names" + Type-valued fields in extension YAML use [type expressions](../types/type_parsing.md), such as `string`, `binary`, and `fixedbinary<16>`. Signature short names are only used when constructing function signatures, such as `concat:str` or `hash:vbin`. + #### Variadic Functions For variadic functions, the variadic argument is included *once* in the argument signature. From 7038265d399367ddd7d5d837f9cc430578f08cf2 Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:15:56 -0400 Subject: [PATCH 6/9] docs: keep schema type description unchanged --- text/simple_extensions_schema.yaml | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/text/simple_extensions_schema.yaml b/text/simple_extensions_schema.yaml index 77e1a6081..199872ca1 100644 --- a/text/simple_extensions_schema.yaml +++ b/text/simple_extensions_schema.yaml @@ -75,10 +75,7 @@ properties: $defs: type: - description: >- - String values are Substrait type expressions parsed by grammar/SubstraitType.g4. - Use type expression names here, such as string, binary, or fixedbinary<16>. - Do not use function signature short names such as str, vbin, or fbin. + description: String values are Substrait type expressions parsed by grammar/SubstraitType.g4. oneOf: - type: string # any data type - type: object # syntactic sugar for a non-nullable named struct From e3debe6ce55ca3f9bf7beef59a795d8c05c0aa60 Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:16:42 -0400 Subject: [PATCH 7/9] docs: explain type short name columns --- site/docs/extensions/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index 813d18b5b..344268459 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -89,6 +89,8 @@ A function signature uniquely identifies a function implementation within a sing #### Type Short Names +The first column lists the type expression used in extension YAML. The second column lists the corresponding short name used in function signatures. + | YAML Type Expression | Signature Short Name | |---------------------------------|----------------------| | Enumeration | req | From fceb71a74b70d969cd16b4bfec9dbd8db112a1c3 Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:20:15 -0400 Subject: [PATCH 8/9] docs: move type expression note to short names section --- site/docs/extensions/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index 344268459..ab131f54f 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -74,10 +74,7 @@ The resulting function signatures look like: `:_ argument-signature = short-arg-type *("_" short-arg-type) ``` -Argument types (`short_arg_type`) are encoded using the Type Short Names given below. - -!!! note "Type expressions vs. signature short names" - Type-valued fields in extension YAML use [type expressions](../types/type_parsing.md), such as `string`, `binary`, and `fixedbinary<16>`. Signature short names are only used when constructing function signatures, such as `concat:str` or `hash:vbin`. +Argument types (`short_arg_type`) are encoded using the [Type Short Names](#type-short-names) given below. #### Variadic Functions @@ -89,6 +86,9 @@ A function signature uniquely identifies a function implementation within a sing #### Type Short Names +!!! note "Type expressions vs. signature short names" + Type-valued fields in extension YAML use [type expressions](../types/type_parsing.md), such as `string`, `binary`, and `fixedbinary<16>`. Signature short names are only used when constructing function signatures, such as `concat:str` or `hash:vbin`. + The first column lists the type expression used in extension YAML. The second column lists the corresponding short name used in function signatures. | YAML Type Expression | Signature Short Name | From e1876e18ee1b1165f61f7de20775d1a08b755576 Mon Sep 17 00:00:00 2001 From: Ben Bellick Date: Mon, 1 Jun 2026 10:21:57 -0400 Subject: [PATCH 9/9] docs: link type syntax grammar in this repo --- site/docs/types/type_parsing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/docs/types/type_parsing.md b/site/docs/types/type_parsing.md index d2b472add..32f52c105 100644 --- a/site/docs/types/type_parsing.md +++ b/site/docs/types/type_parsing.md @@ -17,7 +17,7 @@ The components of this expression are: ### Grammars -It is relatively easy in most languages to produce simple parser & emitters for the type syntax. To make that easier, Substrait also includes an [ANTLR grammar](https://github.com/substrait-io/substrait-cpp/blob/main/src/substrait/textplan/parser/grammar/SubstraitPlanParser.g4#L108) to ease consumption and production of types. (The grammar also supports an entire language for representing plans as text.) +It is relatively easy in most languages to produce simple parser & emitters for the type syntax. To make that easier, Substrait also includes an [ANTLR grammar](https://github.com/substrait-io/substrait/blob/main/grammar/SubstraitType.g4) to ease consumption and production of types. ### Structs & Named Structs