Skip to content

feat(metadata-db): add kind column to manifest_files table#1965

Open
LNSD wants to merge 1 commit intomainfrom
lnsd/feat-metadata-db-manifest-kind-column
Open

feat(metadata-db): add kind column to manifest_files table#1965
LNSD wants to merge 1 commit intomainfrom
lnsd/feat-metadata-db-manifest-kind-column

Conversation

@LNSD
Copy link
Contributor

@LNSD LNSD commented Mar 12, 2026

Store dataset kind alongside manifest hash and path so manifests carry their type identity in the database, enabling kind-aware queries and removing runtime kind inference.

  • Add kind TEXT column via migration with 'unknown' default for existing rows
  • Introduce ManifestKind newtype with Cow for zero-copy database operations
  • Replace NonEmptyString with typed DatasetKind enum and ManifestHeader for dispatch
  • Convert if-else kind matching to exhaustive match on DatasetKind in handlers
  • Remove UnsupportedDatasetKind error variant (now caught at deserialization boundary)

@LNSD LNSD requested review from Theodus, leoyvens and shiyasmohd March 12, 2026 15:08
@LNSD LNSD self-assigned this Mar 12, 2026
@LNSD LNSD force-pushed the lnsd/feat-metadata-db-manifest-kind-column branch from b9dbe82 to 763e5fe Compare March 12, 2026 15:16
-- Defaults to 'unknown' for existing rows.
-- =============================================================

ALTER TABLE manifest_files ADD COLUMN kind TEXT NOT NULL DEFAULT 'unknown';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first of 3 PRs that will require either metadata DB fiddling or redeploying the manifests to upgrade to the latest version

@LNSD LNSD force-pushed the lnsd/feat-metadata-db-manifest-kind-column branch from 763e5fe to 893795e Compare March 12, 2026 15:19
Store dataset kind alongside manifest hash and path so manifests carry their type identity in the database, enabling kind-aware queries and removing runtime kind inference.

- Add `kind` TEXT column via migration with 'unknown' default for existing rows
- Introduce `ManifestKind` newtype with Cow<str> for zero-copy database operations
- Replace `NonEmptyString` with typed `DatasetKind` enum and `ManifestHeader` for dispatch
- Convert if-else kind matching to exhaustive match on `DatasetKind` in handlers
- Remove `UnsupportedDatasetKind` error variant (now caught at deserialization boundary)

Signed-off-by: Lorenzo Delgado <lorenzo@edgeandnode.com>
@LNSD LNSD force-pushed the lnsd/feat-metadata-db-manifest-kind-column branch from 893795e to a1dedb8 Compare March 12, 2026 15:31
Copy link
Member

@Theodus Theodus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants