Skip to content

feat(avro): enable snappy codec support#2573

Merged
CTTY merged 2 commits into
apache:mainfrom
romusz:proman-260602-avro-enable-snappy-codec
Jun 3, 2026
Merged

feat(avro): enable snappy codec support#2573
CTTY merged 2 commits into
apache:mainfrom
romusz:proman-260602-avro-enable-snappy-codec

Conversation

@romusz
Copy link
Copy Markdown
Contributor

@romusz romusz commented Jun 3, 2026

  • Which issue does this PR close?

Error: failed to load current manifest files for ...

Caused by:
    0: DataInvalid => Failure in conversion with avro, source: Codec 'snappy' is not supported/enabled
    1: Codec 'snappy' is not supported/enabled

What changes are included in this PR?

Some producers compress manifests with snappy instead of zstd - this PR just enables Snappy feature in apache-avro

Are these changes tested?

  • external integration tests on snappy-compressed manifest files

@xanderbailey
Copy link
Copy Markdown
Contributor

Thanks for the contribution, a test for this would be great!

@romusz
Copy link
Copy Markdown
Contributor Author

romusz commented Jun 3, 2026

I've just added 2 synth tests, pls let me know if you'd like testdata as well.

romusz added 2 commits June 3, 2026 19:00
- Some producers compress manifests with snappy instead of zstd
- Add synthetic Snappy Avro coverage for
  - manifest lists through ManifestList::parse_with_version
  - V2 data and delete manifests through Manifest::parse_avro
@romusz romusz force-pushed the proman-260602-avro-enable-snappy-codec branch from 0f0f25d to f95e7cb Compare June 3, 2026 23:10
@romusz
Copy link
Copy Markdown
Contributor Author

romusz commented Jun 3, 2026

rebased to resolve conflicts with the current main

Copy link
Copy Markdown
Collaborator

@CTTY CTTY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

This is also related to #1851

@CTTY CTTY merged commit 1f3934e into apache:main Jun 3, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Manifests compressed with Snappy cause load failures

3 participants