fix: apply the left side schema on the right side in set expressions#21052
Open
gruuya wants to merge 1 commit intoapache:mainfrom
Open
fix: apply the left side schema on the right side in set expressions#21052gruuya wants to merge 1 commit intoapache:mainfrom
gruuya wants to merge 1 commit intoapache:mainfrom
Conversation
584cdb4 to
706f8f8
Compare
706f8f8 to
6669e8b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
DataFusion requires all projected expressions to have unique names during planning, so it doesn't support select 0, 0 for instance.
However this shouldn't be an issue when this is just a sub-SELECT in a larger query which does abide by this rule. For example a set expression (UNION, EXCEPT, INTERSECT) query should only require the first SELECT to provide a unique schema, and that should be sufficient.
Furthermore, this requirement is even more redundant, since all field name/aliases other than those in the first SELECT are discarded anyway.
What changes are included in this PR?
SqlToRel::select_to_planpop the schema and pass it down toproject_with_validation_and_schemafunction inLogicalPlanBuilderto properly alias themThe benefit of this approach compared to #20819 is that wildcards are unwrapped and we can properly handle them as well.
The downside is that we need to thread the left schema via the planner context now.
Are these changes tested?
Yes, there are unit tests and SLTs.
Are there any user-facing changes?
New method in
LogicalPlanBuildercalledproject_with_validation_and_schemawhich will alias the projection with the provided schema.