Restore Sort unparser guard for correct ORDER BY placement#20658
Restore Sort unparser guard for correct ORDER BY placement#20658krinart wants to merge 4 commits intoapache:mainfrom
Conversation
| // the inner Projection's raw expressions. The optimizer may add | ||
| // sort expressions to the inner Projection without aliases, while | ||
| // the Sort node's expressions carry aliases from the original plan. | ||
| let mut expr = sort.expr.clone(); |
There was a problem hiding this comment.
I think this is the same as https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.Expr.html#method.unalias_nested
There was a problem hiding this comment.
It apparently isn't quite the same, but I am not sure if the different matters. Let's leave it like this
There was a problem hiding this comment.
I tried to use it here - but test_complex_order_by_with_grouping test failed.
|
@krinart -- this PR also has some conflicts that need to be resolved Maybe you can also address the comment as well? I would like to backport this to the 52 branch for the next patch release
|
|
I also filed #20905 to track this issue so we can merge it back to 52 |
|
We are trying to prepare a 52 and 53 release, and since this issue is a regression I would like to fix it today. I can't push to your branch (as it is in the spiceai fork) so I will likely open a new PR to get it ready |
|
Here is a proposed fix: #20923 |
|
No worries, thank you @krinart -- I reopened this PR and as it is approved I'll merge it when it passes |

Which issue does this PR close?
Restore the
already_projected()guard in the Sort case ofselect_to_sql_recursively. Without it, queries with ORDER BY expressions generate invalid SQL when the Sort node follows an outer Projection.Problem
Two regressions in the DuckDB federation unparser after upgrading to DataFusion 52:
SELECT "SearchPhrase" ... ORDER BY to_timestamp("EventTime") LIMIT 10produce ORDER BY outside the subquery, referencing table names ("hits") that are out of scope.GROUPING()expressions loses thederived_sortsubquery alias, placing sort references outside their valid scope.Root Cause
DF52's optimizer merges
LimitintoSortas a fetch parameter, changing the plan shape fromProjection → Limit → Sort → ...toProjection → Sort(fetch) → ....The Sort case previously had a guard that wrapped the sort into aderived_sortsubquery whenalready_projected()was true. This guard was removed in DF52, causing ORDER BY and LIMIT to be placed on the outer query where inner table references are invalid.Fix
Re-add the guard at the top of the Sort match arm in select_to_sql_recursively: