fix: NUMERIC sign loss in Display + Node bindings decode (#84)#86
Merged
StefanSteiner merged 1 commit intoMay 29, 2026
Merged
Conversation
Fixes two related but distinct bugs that surface together as wrong NUMERIC values in MCP query results and Node bindings: 1. **Core: `Numeric::Display` dropped the sign for values in (-1, 0).** The previous impl derived the sign from `int_part = value / divisor`, which is `0` for sub-unit magnitudes (and `0` prints without a sign); the fractional part was `(value % divisor).abs()` (sign stripped). So `Numeric::new(-5000, 4).to_string()` returned `"0.5000"` instead of `"-0.5000"`. The fix computes the sign explicitly and formats the magnitude via `unsigned_abs()`. This silently flipped the sign of any correlation, 0-1 index, or regression residual that crossed the stringify path. `unsigned_abs()` also avoids the `i128::MIN` overflow panic that `.abs()` would have hit. The MCP `query` tool (`row_value_to_json`) serializes NUMERIC via `to_string()` then `parse::<f64>()`, so `CAST(-0.5 AS numeric(10,4))` was returning JSON `0.5` instead of `-0.5`. 2. **Node bindings: `getF64` was decoding NUMERIC as garbage / NaN.** This is a separate root cause from tableau#84's Display bug, fixed here for the same test surface. `extract_row` and `extract_chunk_columnar` were calling `row.get_f64(i)` for `SqlType::Numeric` columns. `get_f64` only handles native f32/f64 wire types; for NUMERIC it reinterpreted the unscaled-integer bytes as IEEE-754 → garbage values or NaN, regardless of sign. Switched to schema-aware `row.get_numeric(i)`, which honors the column scale and decodes the 8/16-byte / Arrow Decimal wire forms correctly. Added a `CellValue::Numeric(Numeric)` variant; `getString` returns the exact decimal text (preserving scale and sign), `getFloat64` returns the lossy-but-correct double, and `getInt32`/`getInt64` return the truncated integer. `getBigInt` on a `NUMERIC(p, 0)` column now preserves the full unscaled `i128` value (use it instead of `getInt64` for integer NUMERIC values above `Number.MAX_SAFE_INTEGER`); on `NUMERIC(p, scale>0)` it returns `null` rather than silently truncating. The columnar fast path surfaces NUMERIC as correct `f64` values. ## Test coverage - Core: `test_numeric_display_negative_sign_preserved` — covers -0.5, -0.999, -0.0001, -1.5, 0, +0.5, scale-0 negative, and `i128::MIN` (no panic). - MCP: `engine_negative_sub_unit_numeric_keeps_sign` — exercises the exact reproduction from the issue body via `execute_query_to_json`. - Node smoke (section 18b): row-wise `getString`/`getFloat64`, AVG aggregate, columnar fast path, `getBigInt` precision preservation for `NUMERIC(38,0)` values above `Number.MAX_SAFE_INTEGER`, `getBigInt` returning null on `NUMERIC(p, scale>0)`. Adversarial review (line-level + architect in parallel) caught the missing `getBigInt` Numeric arm and the smoke-test gap; both addressed before commit. The Cargo.lock regen in `hyperdb-api-node/package-lock.json` removes stale `version: "0.1.3"` and `optionalDependencies` entries that violate the version-consistency CI guard (npm package versions are injected at publish time, not tracked in source). Closes tableau#84.
f2af85a to
a9270a7
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two related but distinct bugs surfaced by #84. They share the same test surface (wrong NUMERIC values out of MCP / Node) but have independent root causes; bundling here for delivery, but flagging clearly so future bisect-readers don't conflate them.
Bug 1 — Core:
Numeric::Displaydrops sign for sub-unit negatives (#84 root cause)impl fmt::Display for Numericinhyperdb-api-core/src/types/special.rsderived the sign fromint_part = value / divisor. For|value| < divisor(i.e.(-1, 0)),int_partis0— and0prints without a sign. The fractional part used.abs(), which stripped the sign too. Result:Numeric::new(-5000, 4).to_string()returned"0.5000"instead of"-0.5000".This silently flipped the sign of any correlation, 0–1 index, or regression residual that crossed the stringify path. The MCP
querytool (row_value_to_json) serializes NUMERIC viato_string()thenparse::<f64>(), soCAST(-0.5 AS numeric(10,4))was returning JSON0.5instead of-0.5.Fix: compute the sign explicitly, format the magnitude via
unsigned_abs(). The latter also removes a latenti128::MIN.abs()overflow panic.Bug 2 — Node bindings: NUMERIC decoded as garbage / NaN (separate root cause)
hyperdb-api-node/src/result.rs:extract_rowandcolumnar.rs:extract_chunk_columnarwere callingrow.get_f64(i)forSqlType::Numericcolumns.get_f64only handles native f32/f64 wire types; for NUMERIC it reinterpreted the unscaled-integer bytes as IEEE-754 → garbage values orNaN, regardless of sign. This is a different and worse failure than #84's Display bug: every NUMERIC cell was wrong, not just sub-unit negatives.Fix: switch to schema-aware
row.get_numeric(i), which honors the column scale and dispatches on wire form. AddedCellValue::Numeric(Numeric)variant. The accessor surface:getStringgetFloat64f64, possibly lossy above 15 sig digitsgetInt32/getInt64to_f64() as i32/i64getBigIntNUMERIC(p, 0): fulli128precision; forNUMERIC(p, scale>0):nullgetFloat64Columnf64(was garbage / NaN)getBigIntonNUMERIC(p, 0)is the precision-preserving path for big integer NUMERIC values aboveNumber.MAX_SAFE_INTEGER(i64::MAXround-trips exactly).Why bundle them
The Display bug surfaces in MCP query JSON; the Node decode bug surfaces in the Node bindings. Both produce "wrong NUMERIC values" symptomatically, and they share a test surface (subtests in the smoke test cover both paths). Splitting into two PRs would have meant duplicated test setup and a confusing review of the second PR (the Node fix relies on the core Display being correct for
getStringto be exact). The commit body and CHANGELOGs treat them as two distinct findings; the issue link annotation in the Node CHANGELOG is "Relates to #84" rather than "Fixes #84" to keep attribution honest.Test coverage
test_numeric_display_negative_sign_preserved) — covers -0.5, -0.999, -0.0001, -1.5, 0, +0.5, scale-0 negative, andi128::MIN(no panic).engine_negative_sub_unit_numeric_keeps_sign) — runs the exact reproduction from the issue body throughexecute_query_to_json:CAST(-0.5 AS numeric(10,4)),CAST(-0.999 AS numeric(10,4)),CAST(-1.5 AS numeric(10,4))(control: was working),CAST(-0.5 AS double precision)(control: float path).getString/getFloat64/getInt32/getInt64/getBigInt, AVG aggregate, columnar fast path, high-precision (38-digit) values,getBigIntprecision preservation forNUMERIC(38,0)values aboveNumber.MAX_SAFE_INTEGER,getBigIntreturning null on non-zero-scale NUMERIC.Adversarial review
Both reviewers (line-level + architect) ran in parallel against the uncommitted diff. Line-level came back clean. Architect surfaced two MAJOR findings the line-level missed:
get_big_inthad noNumericarm — the CHANGELOG claimed "NUMERIC columns are now decoded correctly" butgetBigIntsilently returnednullon every NUMERIC cell. Fixed before commit.getInt32/getInt64/getBigInton NUMERIC. Added.Other architect findings deferred:
n.to_f64() as i32round-trip lossy forNUMERIC(38,0)above 2^53 — mitigated by thegetBigIntfix;getInt32/getInt64are documented narrowing paths.Verification
cargo build --workspace --all-targets✅cargo clippy --workspace --all-targets -- -D warnings✅cargo test --workspace✅cargo fmt --check✅cargo doc --workspace --no-deps✅ — 6 warnings (= post-Flatten Error type to canonical M-ERRORS shape (drop Client wrapper, Box<dyn> source, Option<ErrorKind>) #70 baseline)npm run build:debug && npm test(full Node smoke suite, 19 sections, end-to-end against local hyperd) ✅Note on
package-lock.jsonThe diff to
hyperdb-api-node/package-lock.jsonis deletions only — it removes staleversion: "0.1.3"andoptionalDependenciesentries that violate theversion-consistencyCI guard (ci.yml:259-314). npm package versions are injected at publish time bynpm-build-publish.yml; they must not be tracked in source.Release plan
This is a real bug fix that affects published packages on crates.io and npm. Recommend cutting v0.3.1 as a patch release after this merges. With
bump-minor-pre-major: trueinrelease-please-config.json(from #79) and thefix:commit prefix, release-please will propose a clean 0.3.0 → 0.3.1 bump. Then follow the manual tag procedure (PR #79 / #83 docs) to ship.Closes #84.