refactor: unify SQL planning for ORDER BY, HAVING, DISTINCT, etc

[lichuang]

Which issue does this PR close?

• Closes Unify SQL planning for ORDER BY, HAVING, DISTINCT, etc #10326

This PR refactors ORDER BY planning to use the merged schema approach (similar to HAVING), unifying the SQL planning logic and simplifying the generated ex
ecution plans.

Currently, DataFusion has two different code paths for handling ORDER BY:

1. add_missing_columns (in LogicalPlanBuilder::sort_with_limit): Traverses the plan tree looking for Projection nodes and adds missing columns to them
2. Merged Schema approach (used by HAVING): Uses a merged schema (SELECT list + FROM clause) to resolve expressions and directly adds missing columns to the SELECT list

Having both paths coexist leads to:

• Complex and hard-to-maintain code
• Non-intuitive handling of simple queries like SELECT x FROM foo ORDER BY y
• Generated execution plans with unnecessary subquery wrapping

Solution

Implement the approach proposed in #10326: handle ORDER BY similarly to HAVING by using the merged schema and adding missing columns directly to the SELECT
list instead of traversing the plan tree.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[lichuang]

@alamb @jonahgao

[alamb]

This seems to add more code (rather than unify the code paths as the PR comment suggests) 🤔

[alamb]

In other words, this PR seems to add more code paths, when the idea was to reduce the code / duplication