Skip to content

[Correctness Testing] All Spark Versions - Add Parquet related tests #2172

@ShreyeshArangath

Description

@ShreyeshArangath

Is your feature request related to a problem? Please describe.

Extend Parquet correctness tests (originally added for Spark 3.3 in #1816) to all other supported Spark versions.

Parent epic: #1745

Version-Specific Notes

  • Spark 3.1: 8 Parquet suites unavailable (ParquetColumnIndexSuite, ParquetFieldIdIOSuite, ParquetFieldIdSchemaSuite, ParquetRebaseDatetimeSuite, ParquetRebaseDatetimeV1Suite, ParquetRebaseDatetimeV2Suite, ParquetVectorizedSuite)
  • Spark 3.2: 3 Parquet suites unavailable (ParquetFieldIdIOSuite, ParquetFieldIdSchemaSuite, ParquetVectorizedSuite)
  • Spark 3.4, 3.5, 4.0, 4.1: All 32 suites available

Versions

  • Spark 3.1
  • Spark 3.2
  • Spark 3.4
  • Spark 3.5
  • Spark 4.0
  • Spark 4.1

Describe the solution you'd like

Implement correctness testing

Describe alternatives you've considered

N/A

Additional context

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions