[ABA-30] fix(vortex-sparse): normalize patches dtype before slot construction#4
Open
abnobdoss wants to merge 2 commits into
Open
[ABA-30] fix(vortex-sparse): normalize patches dtype before slot construction#4abnobdoss wants to merge 2 commits into
abnobdoss wants to merge 2 commits into
Conversation
…-normalized slot dtype `Sparse::try_new_from_patches` captures the slot `ArrayRef` from the caller's `Patches` before `normalize_patches_dtype` runs, so when the caller supplies `Patches` whose `values().dtype()` differs in nullability from `fill_value.dtype()`, the resulting `SparseArray`'s `PATCH_VALUES` slot disagrees with the array's declared dtype. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
…eArray slots `Sparse::try_new_from_patches` called `SparseData::make_slots(&patches)` on the caller's `Patches` before `SparseData::from_patches` ran `normalize_patches_dtype`. When `fill_value.dtype()` differed from `patches.values().dtype()` (e.g. in nullability), the slot `ArrayRef` captured the un-normalized values while the array's declared dtype and `SparseData` reflected the normalized fill dtype. The constructor commits the inconsistent parts via `Array::from_parts_unchecked`, so the mismatch is committed silently and any consumer that reads patches via `SparseExt::patches` (which rebuilds `Patches` from the slots) sees the wrong nullability. Reorder so `normalize_patches_dtype` runs before `make_slots`, then call `from_patches_unchecked` since normalization has already happened. The sibling constructors (`Sparse::try_new` and `SparseData::from_patches`) remain unchanged because they only have one source of truth for the values dtype. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sparse::try_new_from_patchescalledSparseData::make_slots(&patches)on the caller'sPatchesbeforeSparseData::from_patchesrannormalize_patches_dtype. Whenfill_value.dtype()differed frompatches.values().dtype()in nullability, the slotArrayRefcaptured the un-normalized values while the array's declared dtype andSparseDatareflected the normalized fill dtype. The constructor commits the inconsistent parts viaArray::from_parts_unchecked, so the mismatch was committed silently — and any consumer that reads patches viaSparseExt::patches(which rebuildsPatchesfrom the slots) saw the wrong nullability.Fix: reorder so
normalize_patches_dtyperuns beforemake_slots, then callfrom_patches_uncheckedsince normalization has already happened. The sibling constructors (Sparse::try_newandSparseData::from_patches) are unchanged.Two-commit layout: the first commit adds a failing regression test; the second applies the reorder.
pub fn try_new_from_patches(patches: Patches, fill_value: Scalar) -> VortexResult<SparseArray> { let dtype = fill_value.dtype().clone(); let len = patches.array_len(); + let patches = SparseData::normalize_patches_dtype(patches, &fill_value)?; let slots = SparseData::make_slots(&patches); - let data = SparseData::from_patches(&patches, fill_value)?; + let data = SparseData::from_patches_unchecked(&patches, fill_value); Ok(unsafe { Array::from_parts_unchecked(ArrayParts::new(Sparse, dtype, len, data).with_slots(slots)) }) }Linear
ABA-30 —
Sparse::try_new_from_patchesattaches slots from un-normalized patches.Validation
vortex_sparse::test::issue_aba30_sparse_slot_dtype_matches_declared_dtypefails ondevelop(slot dtypePrimitive(I32, NonNullable)vs array dtypePrimitive(I32, Nullable)) and passes after the reorder.cargo test -p vortex-sparse— 80 passed, 0 failed.cargo clippy -p vortex-sparse --all-targets --all-features— clean.cargo +nightly fmt --all -- --check— clean.public-api.lockunaffected.Attribution
Co-authored by Claude Opus 4.7 (1M context). Signed-off-by DCO on both commits.