Skip to content

[AURON #2013] Auto-lock Scala version in Spark profiles.#2014

Open
slfan1989 wants to merge 1 commit intoapache:masterfrom
slfan1989:auron-2013
Open

[AURON #2013] Auto-lock Scala version in Spark profiles.#2014
slfan1989 wants to merge 1 commit intoapache:masterfrom
slfan1989:auron-2013

Conversation

@slfan1989
Copy link
Contributor

@slfan1989 slfan1989 commented Feb 17, 2026

Which issue does this PR close?

Closes #2013

Rationale for this change

Currently, when developing in IntelliJ IDEA, users need to manually select both a Spark profile (e.g., spark-3.5) and a Scala profile (e.g., scala-2.12) to ensure correct compilation. This two-step configuration is error-prone and can lead to compatibility issues if mismatched.

This change simplifies the workflow by automatically locking the Scala version and compiler configuration when a Spark profile is selected.

What changes are included in this PR?

  1. Scala version properties:
  • Spark 3.x profiles: scalaVersion=2.12, scalaLongVersion=2.12.18
  • Spark 4.x profiles: scalaVersion=2.13, scalaLongVersion=2.13.17
  1. scala-maven-plugin configuration:
  • Spark 3.x: Added semanticdb-scalac + paradise compiler plugins
  • Spark 4.x: Added -Ymacro-annotations args + semanticdb-scalac plugin (without paradise)
  • Used combine.self="override" to prevent configuration merging with base settings

Are there any user-facing changes?

Yes. Users now only need to select a Spark profile (e.g., spark-3.5) in IDEA, and the corresponding Scala version will be automatically configured. No need to manually select separate scala-2.12 or scala-2.13 profiles.

How was this patch tested?

Verified in IntelliJ IDEA by:

  1. Selecting only spark-3.5 profile → Scala 2.12.18 correctly configured
  2. Selecting only spark-4.0 profile → Scala 2.13.17 correctly configured
  3. No need to select separate scala-2.12 or scala-2.13 profiles

Signed-off-by: slfan1989 <slfan1989@apache.org>
@github-actions github-actions bot added the build label Feb 17, 2026
@slfan1989
Copy link
Contributor Author

  • When we select Spark 3.5, we can see that IntelliJ IDEA automatically switches to it.
Image
  • When we select Spark 4.1, we can see that IntelliJ IDEA automatically switches to it.
Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-lock Scala version in Spark profiles

1 participant