Skip to content

Add FULL OUTER JOIN support#38340

Merged
roji merged 1 commit into
mainfrom
roji/full-outer-join
Jun 2, 2026
Merged

Add FULL OUTER JOIN support#38340
roji merged 1 commit into
mainfrom
roji/full-outer-join

Conversation

@roji
Copy link
Copy Markdown
Member

@roji roji commented Jun 1, 2026

Fixes #37633.

Adds translation support for the new Queryable.FullJoin LINQ operator (dotnet/runtime#124787), mirroring the existing LeftJoin/RightJoin support added in #35451.

Summary

  • Register QueryableMethods.FullJoin and dispatch it through the query pipeline (NavigationExpandingExpressionVisitor, QueryableMethodTranslatingExpressionVisitor).
  • New FullJoinExpression SQL expression, with SelectExpression handling (both sides nullable), QuerySqlGenerator.VisitFullJoin, and AddFullJoin APIs.
  • Providers: relational translation via TranslateFullJoin; InMemory and Cosmos report the operator as untranslatable.

Notes

  • API shape: Unlike LeftJoin/RightJoin (which expose separate no-comparer and comparer overloads), FullJoin has a single overload with an optional trailing IEqualityComparer<TKey>. Every call therefore emits a 6-arg expression with a comparer argument (compiler-supplied null when omitted). We only translate the null-comparer case; a custom comparer produces a translation failure.
  • Funcletizer: Because the comparer is a top-level operator argument, the default null would normally be parameterized — hiding from the translator whether a (non-translatable) custom comparer was supplied. IsParameterParameterizable now excludes IEqualityComparer<> arguments from parameterization, treating them like [NotParameterized]. This is the first single-overload-optional-comparer operator EF translates.

Latent RIGHT JOIN correctness fix (discovered while adding FULL JOIN)

SelectExpression.AddJoin previously left an outer Predicate as a post-join WHERE for nullable-outer joins. Once the outer side becomes nullable (RIGHT/FULL JOIN), a post-join WHERE on an outer column wrongly drops the unmatched rows (where the outer is NULL), collapsing the join toward an inner join. The outer is now pushed into a subquery (so its predicate is applied before the join) for RightJoin/FullJoin when it has a predicate.

This is the same invariant already handled for the inner side of LEFT JOIN; it was simply never applied to the nullable outer side. The fix corrects a pre-existing, latent RIGHT JOIN bug as well as FULL JOIN.

Consequences:

  • Two pre-existing NorthwindBulkUpdates RIGHT JOIN tests had their behavior corrected. Update_with_RightJoin's verification now correctly sees the unmatched right-side rows as null outer entities; its assertion was tightened to assert that shape explicitly (all 8 F-customers returned, 6 unmatched/null, 2 matched and updated). Delete_with_RightJoin is unchanged at runtime (the deleted target is the nullable side, so affected-row counts are identical) — only its SQL baseline changed.
  • SQL baselines updated for SqlServer and Sqlite.

Tests

New NorthwindJoinQuery tests (basic full join, unmatched rows on both sides, custom-comparer translation failure, plus a direct RightJoin_with_filtered_outer regression test) with provider overrides. Full SqlServer + Sqlite Query and BulkUpdates suites pass.

Depends on an SDK bump (global.json) to a preview that contains the FullJoin operator.

Implement translation of the new LINQ `Queryable.FullJoin` operator to SQL
`FULL JOIN`, mirroring the existing RightJoin support but making BOTH the outer
and inner sides nullable.

Core:
- Register `QueryableMethods.FullJoin` (single 6-param overload with an optional
  `IEqualityComparer<TKey>`) and dispatch it in the navigation-expansion and
  query-translation visitors. A non-null custom comparer is rejected as
  untranslatable, matching Left/RightJoin.
- Never parameterize `IEqualityComparer<>`/`IComparer<>` operator arguments in the
  funcletizer (treat them like `[NotParameterized]`): a comparer is structural, not
  a value, so it must be evaluated as a constant. This lets the translator detect
  (and reject) a non-null FullJoin comparer instead of seeing a query parameter.

Relational:
- Add `FullJoinExpression`, `SelectExpression.AddFullJoin`, `JoinType.FullJoin`
  (both sides nullable, never prunable/to-one) and `QuerySqlGenerator.VisitFullJoin`.
- Fix a latent correctness bug shared with RightJoin: when the outer side becomes
  nullable, an outer `WHERE` predicate must be pushed into a subquery before the
  join, otherwise it is emitted post-join and incorrectly filters out the
  unmatched inner rows.

Providers: InMemory and Cosmos report FullJoin as untranslatable.

Bump the SDK in global.json to a preview.5 build containing the FullJoin operator.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@roji roji force-pushed the roji/full-outer-join branch from d4376cc to e4cd707 Compare June 1, 2026 18:07
@roji roji marked this pull request as ready for review June 1, 2026 20:04
@roji roji requested review from a team and AndriySvyryd as code owners June 1, 2026 20:04
Copilot AI review requested due to automatic review settings June 1, 2026 20:04
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds translation support for the new Queryable.FullJoin LINQ operator, mirroring the existing LeftJoin/RightJoin infrastructure. Introduces a new FullJoinExpression, wires it through the query pipeline, and provides provider-specific handling for Relational (translated), InMemory and Cosmos (reported as untranslatable). Also fixes a latent RIGHT JOIN correctness bug where an outer-side predicate was incorrectly applied as a post-join WHERE, which would drop unmatched rows once the outer became nullable.

Changes:

  • New FullJoinExpression and AddFullJoin/TranslateFullJoin/VisitFullJoin plumbing across SelectExpression, QuerySqlGenerator, and the relational/in-memory/cosmos translators, plus QueryableMethods.FullJoin registration and dispatch in NavigationExpandingExpressionVisitor/QueryableMethodTranslatingExpressionVisitor.
  • ExpressionTreeFuncletizer now refuses to parameterize IEqualityComparer<> arguments, so the compiler-supplied default null on FullJoin is preserved as a constant and translation can distinguish a translatable from a non-translatable comparer.
  • SelectExpression.AddJoin now pushes the outer into a subquery when it has a Predicate for RightJoin/FullJoin, fixing a latent RIGHT JOIN bug; SqlServer/Sqlite baselines and one tightened Update_with_RightJoin assertion are updated accordingly, plus new NorthwindJoinQueryTestBase tests cover full join and the right-join regression.
Show a summary per file
File Description
src/EFCore/Query/QueryableMethods.cs Registers the Queryable.FullJoin MethodInfo (6-arg overload).
src/EFCore/Query/QueryableMethodTranslatingExpressionVisitor.cs Dispatches FullJoin (null-comparer only) and adds abstract TranslateFullJoin.
src/EFCore/Query/Internal/NavigationExpandingExpressionVisitor.cs Handles FullJoin in navigation expansion; rebuilds the call with trailing null comparer and marks both shapers nullable.
src/EFCore/Query/Internal/ExpressionTreeFuncletizer.cs Excludes IEqualityComparer<> parameters from parameterization.
src/EFCore/EFCore.baseline.json API baseline additions for FullJoin/TranslateFullJoin.
src/EFCore.Relational/Query/SqlExpressions/FullJoinExpression.cs New FullJoinExpression mirroring RightJoinExpression.
src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs Adds JoinType.FullJoin, both-sides nullable handling, identifier nullability, and the outer-predicate pushdown fix for RIGHT/FULL JOIN.
src/EFCore.Relational/Query/QuerySqlGenerator.cs Emits FULL JOIN SQL via VisitFullJoin.
src/EFCore.Relational/Query/RelationalQueryableMethodTranslatingExpressionVisitor.cs Implements relational TranslateFullJoin.
src/EFCore.Relational/EFCore.Relational.baseline.json API baseline additions for the new expression and AddFullJoin/VisitFullJoin/TranslateFullJoin members.
src/EFCore.InMemory/Query/Internal/InMemoryQueryableMethodTranslatingExpressionVisitor.cs Returns null (untranslatable) for FullJoin.
src/EFCore.Cosmos/Query/Internal/CosmosQueryableMethodTranslatingExpressionVisitor.cs Reports FullJoin as untranslatable with the cross-document error.
test/EFCore.Specification.Tests/Query/NorthwindJoinQueryTestBase.cs New FullJoin, unmatched-both-sides, custom-comparer, and RightJoin_with_filtered_outer tests.
test/EFCore.Specification.Tests/BulkUpdates/NorthwindBulkUpdatesTestBase.cs Tightens Update_with_RightJoin assertion to verify null outer entries for unmatched rows.
test/EFCore.InMemory.FunctionalTests/Query/NorthwindJoinQueryInMemoryTest.cs Overrides new tests with AssertTranslationFailed.
test/EFCore.SqlServer.FunctionalTests/Query/NorthwindJoinQuerySqlServerTest.cs SqlServer SQL baselines for the new join tests.
test/EFCore.SqlServer.FunctionalTests/BulkUpdates/NorthwindBulkUpdatesSqlServerTest.cs Updated SqlServer SQL baselines reflecting the outer-predicate pushdown.
test/EFCore.Sqlite.FunctionalTests/BulkUpdates/NorthwindBulkUpdatesSqliteTest.cs Updated Sqlite SQL baselines reflecting the outer-predicate pushdown.
global.json Bumps SDK to a preview that contains the FullJoin operator.

Copilot's findings

  • Files reviewed: 19/19 changed files
  • Comments generated: 0

@roji roji merged commit 7ca70b9 into main Jun 2, 2026
15 checks passed
@roji roji deleted the roji/full-outer-join branch June 2, 2026 09:20
@github-actions github-actions Bot added the api-review This PR or issue is introducing public API changes that need to be reviewed label Jun 2, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

API review baseline changes for src/EFCore.Relational/EFCore.Relational.baseline.json

Show diff

The diff below was generated by ApiChief between the base and the PR.

  class Microsoft.EntityFrameworkCore.Query.QuerySqlGenerator : System.Linq.Expressions.ExpressionVisitor
+ virtual Expression VisitFullJoin(FullJoinExpression fullJoinExpression);
  class Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor :
      Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor
+ override ShapedQueryExpression? TranslateFullJoin(ShapedQueryExpression outer, ShapedQueryExpression inner, LambdaExpression outerKeySelector,
+     LambdaExpression innerKeySelector, LambdaExpression resultSelector);
+ class Microsoft.EntityFrameworkCore.Query.SqlExpressions.FullJoinExpression : Microsoft.EntityFrameworkCore.Query.SqlExpressions.PredicateJoinExpressionBase
+ override bool Equals(object? obj);
+ FullJoinExpression(TableExpressionBase table, SqlExpression joinPredicate, bool prunable = false);
+ override int GetHashCode();
+ override void Print(ExpressionPrinter expressionPrinter);
+ override Expression Quote();
+ virtual FullJoinExpression Update(TableExpressionBase table);
+ virtual FullJoinExpression Update(TableExpressionBase table, SqlExpression joinPredicate);
+ virtual FullJoinExpression WithAnnotations(IReadOnlyDictionary<string, IAnnotation> annotations);
  sealed class Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression : Microsoft.EntityFrameworkCore.Query.SqlExpressions.TableExpressionBase
+ Expression AddFullJoin(ShapedQueryExpression innerSource, SqlExpression joinPredicate, Expression outerShaper);
+ void AddFullJoin(SelectExpression innerSelectExpression, SqlExpression joinPredicate);

API review baseline changes for src/EFCore/EFCore.baseline.json

Show diff

The diff below was generated by ApiChief between the base and the PR.

  abstract class Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor : System.Linq.Expressions.ExpressionVisitor
+ abstract ShapedQueryExpression? TranslateFullJoin(ShapedQueryExpression outer, ShapedQueryExpression inner, LambdaExpression outerKeySelector,
+     LambdaExpression innerKeySelector, LambdaExpression resultSelector);
  static class Microsoft.EntityFrameworkCore.Query.QueryableMethods
+ static MethodInfo FullJoin { get; }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api-review This PR or issue is introducing public API changes that need to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support FULL OUTER JOINs

3 participants