[EPIC] Add support for Spark 4.1

[yew1eb]

This epic tracks support for Apache Spark 4.1(Spark Release 4.1.1 ) in Auron (current supported versions: 3.0~3.5).
It is proposed to prioritize Spark 4.1 as the first supported Spark 4.x version to accelerate the Spark 4 compatibility initiative, with the implementation designed for extensibility to enable easy addition of Spark 4.0 support at a later stage if required.

Sub Tasks

• Initial Spark 4.1 Compatibility Support #1956
• Fix Spark 4.1: Catalyst codegen failure in unit tests due to session artifacts isolation #1961
• Support Spark 4.1 Unit Tests [To be added]