Skip to content

[ABA-30] fix(vortex-sparse): normalize patches dtype before slot construction#4

Open
abnobdoss wants to merge 2 commits into
developfrom
fix/aba-30-sparse-normalize-before-slots
Open

[ABA-30] fix(vortex-sparse): normalize patches dtype before slot construction#4
abnobdoss wants to merge 2 commits into
developfrom
fix/aba-30-sparse-normalize-before-slots

Conversation

@abnobdoss
Copy link
Copy Markdown
Owner

Summary

Sparse::try_new_from_patches called SparseData::make_slots(&patches) on the caller's Patches before SparseData::from_patches ran normalize_patches_dtype. When fill_value.dtype() differed from patches.values().dtype() in nullability, the slot ArrayRef captured the un-normalized values while the array's declared dtype and SparseData reflected the normalized fill dtype. The constructor commits the inconsistent parts via Array::from_parts_unchecked, so the mismatch was committed silently — and any consumer that reads patches via SparseExt::patches (which rebuilds Patches from the slots) saw the wrong nullability.

Fix: reorder so normalize_patches_dtype runs before make_slots, then call from_patches_unchecked since normalization has already happened. The sibling constructors (Sparse::try_new and SparseData::from_patches) are unchanged.

Two-commit layout: the first commit adds a failing regression test; the second applies the reorder.

 pub fn try_new_from_patches(patches: Patches, fill_value: Scalar) -> VortexResult<SparseArray> {
     let dtype = fill_value.dtype().clone();
     let len = patches.array_len();
+    let patches = SparseData::normalize_patches_dtype(patches, &fill_value)?;
     let slots = SparseData::make_slots(&patches);
-    let data = SparseData::from_patches(&patches, fill_value)?;
+    let data = SparseData::from_patches_unchecked(&patches, fill_value);
     Ok(unsafe {
         Array::from_parts_unchecked(ArrayParts::new(Sparse, dtype, len, data).with_slots(slots))
     })
 }

Linear

ABA-30 — Sparse::try_new_from_patches attaches slots from un-normalized patches.

Validation

  • New regression test vortex_sparse::test::issue_aba30_sparse_slot_dtype_matches_declared_dtype fails on develop (slot dtype Primitive(I32, NonNullable) vs array dtype Primitive(I32, Nullable)) and passes after the reorder.
  • cargo test -p vortex-sparse — 80 passed, 0 failed.
  • cargo clippy -p vortex-sparse --all-targets --all-features — clean.
  • cargo +nightly fmt --all -- --check — clean.
  • No public API change; public-api.lock unaffected.

Attribution

Co-authored by Claude Opus 4.7 (1M context). Signed-off-by DCO on both commits.

Abanoub Doss and others added 2 commits May 21, 2026 07:58
…-normalized slot dtype

`Sparse::try_new_from_patches` captures the slot `ArrayRef` from the
caller's `Patches` before `normalize_patches_dtype` runs, so when the
caller supplies `Patches` whose `values().dtype()` differs in
nullability from `fill_value.dtype()`, the resulting `SparseArray`'s
`PATCH_VALUES` slot disagrees with the array's declared dtype.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
…eArray slots

`Sparse::try_new_from_patches` called `SparseData::make_slots(&patches)`
on the caller's `Patches` before `SparseData::from_patches` ran
`normalize_patches_dtype`. When `fill_value.dtype()` differed from
`patches.values().dtype()` (e.g. in nullability), the slot `ArrayRef`
captured the un-normalized values while the array's declared dtype and
`SparseData` reflected the normalized fill dtype. The constructor commits
the inconsistent parts via `Array::from_parts_unchecked`, so the
mismatch is committed silently and any consumer that reads patches via
`SparseExt::patches` (which rebuilds `Patches` from the slots) sees the
wrong nullability.

Reorder so `normalize_patches_dtype` runs before `make_slots`, then call
`from_patches_unchecked` since normalization has already happened. The
sibling constructors (`Sparse::try_new` and `SparseData::from_patches`)
remain unchanged because they only have one source of truth for the
values dtype.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant