Skip to content

fix: NUMERIC sign loss in Display + Node bindings decode (#84)#86

Merged
StefanSteiner merged 1 commit into
tableau:mainfrom
StefanSteiner:fix/numeric-sign-loss-84
May 29, 2026
Merged

fix: NUMERIC sign loss in Display + Node bindings decode (#84)#86
StefanSteiner merged 1 commit into
tableau:mainfrom
StefanSteiner:fix/numeric-sign-loss-84

Conversation

@StefanSteiner
Copy link
Copy Markdown
Contributor

Summary

Fixes two related but distinct bugs surfaced by #84. They share the same test surface (wrong NUMERIC values out of MCP / Node) but have independent root causes; bundling here for delivery, but flagging clearly so future bisect-readers don't conflate them.

Bug 1 — Core: Numeric::Display drops sign for sub-unit negatives (#84 root cause)

impl fmt::Display for Numeric in hyperdb-api-core/src/types/special.rs derived the sign from int_part = value / divisor. For |value| < divisor (i.e. (-1, 0)), int_part is 0 — and 0 prints without a sign. The fractional part used .abs(), which stripped the sign too. Result: Numeric::new(-5000, 4).to_string() returned "0.5000" instead of "-0.5000".

This silently flipped the sign of any correlation, 0–1 index, or regression residual that crossed the stringify path. The MCP query tool (row_value_to_json) serializes NUMERIC via to_string() then parse::<f64>(), so CAST(-0.5 AS numeric(10,4)) was returning JSON 0.5 instead of -0.5.

Fix: compute the sign explicitly, format the magnitude via unsigned_abs(). The latter also removes a latent i128::MIN .abs() overflow panic.

Bug 2 — Node bindings: NUMERIC decoded as garbage / NaN (separate root cause)

hyperdb-api-node/src/result.rs:extract_row and columnar.rs:extract_chunk_columnar were calling row.get_f64(i) for SqlType::Numeric columns. get_f64 only handles native f32/f64 wire types; for NUMERIC it reinterpreted the unscaled-integer bytes as IEEE-754 → garbage values or NaN, regardless of sign. This is a different and worse failure than #84's Display bug: every NUMERIC cell was wrong, not just sub-unit negatives.

Fix: switch to schema-aware row.get_numeric(i), which honors the column scale and dispatches on wire form. Added CellValue::Numeric(Numeric) variant. The accessor surface:

Accessor NUMERIC behavior
getString exact decimal text (preserves scale + sign)
getFloat64 correct f64, possibly lossy above 15 sig digits
getInt32 / getInt64 truncated integer via to_f64() as i32/i64
getBigInt for NUMERIC(p, 0): full i128 precision; for NUMERIC(p, scale>0): null
columnar getFloat64Column correct f64 (was garbage / NaN)

getBigInt on NUMERIC(p, 0) is the precision-preserving path for big integer NUMERIC values above Number.MAX_SAFE_INTEGER (i64::MAX round-trips exactly).

Why bundle them

The Display bug surfaces in MCP query JSON; the Node decode bug surfaces in the Node bindings. Both produce "wrong NUMERIC values" symptomatically, and they share a test surface (subtests in the smoke test cover both paths). Splitting into two PRs would have meant duplicated test setup and a confusing review of the second PR (the Node fix relies on the core Display being correct for getString to be exact). The commit body and CHANGELOGs treat them as two distinct findings; the issue link annotation in the Node CHANGELOG is "Relates to #84" rather than "Fixes #84" to keep attribution honest.

Test coverage

  • Core unit test (test_numeric_display_negative_sign_preserved) — covers -0.5, -0.999, -0.0001, -1.5, 0, +0.5, scale-0 negative, and i128::MIN (no panic).
  • MCP integration test (engine_negative_sub_unit_numeric_keeps_sign) — runs the exact reproduction from the issue body through execute_query_to_json: CAST(-0.5 AS numeric(10,4)), CAST(-0.999 AS numeric(10,4)), CAST(-1.5 AS numeric(10,4)) (control: was working), CAST(-0.5 AS double precision) (control: float path).
  • Node smoke test (section 18b) — row-wise getString/getFloat64/getInt32/getInt64/getBigInt, AVG aggregate, columnar fast path, high-precision (38-digit) values, getBigInt precision preservation for NUMERIC(38,0) values above Number.MAX_SAFE_INTEGER, getBigInt returning null on non-zero-scale NUMERIC.

Adversarial review

Both reviewers (line-level + architect) ran in parallel against the uncommitted diff. Line-level came back clean. Architect surfaced two MAJOR findings the line-level missed:

  • get_big_int had no Numeric arm — the CHANGELOG claimed "NUMERIC columns are now decoded correctly" but getBigInt silently returned null on every NUMERIC cell. Fixed before commit.
  • Smoke test missed getInt32/getInt64/getBigInt on NUMERIC. Added.

Other architect findings deferred:

  • Q2: n.to_f64() as i32 round-trip lossy for NUMERIC(38,0) above 2^53 — mitigated by the getBigInt fix; getInt32/getInt64 are documented narrowing paths.
  • Q5, Q7, Q8 — no concerns.

Verification

Note on package-lock.json

The diff to hyperdb-api-node/package-lock.json is deletions only — it removes stale version: "0.1.3" and optionalDependencies entries that violate the version-consistency CI guard (ci.yml:259-314). npm package versions are injected at publish time by npm-build-publish.yml; they must not be tracked in source.

Release plan

This is a real bug fix that affects published packages on crates.io and npm. Recommend cutting v0.3.1 as a patch release after this merges. With bump-minor-pre-major: true in release-please-config.json (from #79) and the fix: commit prefix, release-please will propose a clean 0.3.0 → 0.3.1 bump. Then follow the manual tag procedure (PR #79 / #83 docs) to ship.

Closes #84.

Fixes two related but distinct bugs that surface together as wrong NUMERIC
values in MCP query results and Node bindings:

1. **Core: `Numeric::Display` dropped the sign for values in (-1, 0).**
   The previous impl derived the sign from `int_part = value / divisor`,
   which is `0` for sub-unit magnitudes (and `0` prints without a sign);
   the fractional part was `(value % divisor).abs()` (sign stripped). So
   `Numeric::new(-5000, 4).to_string()` returned `"0.5000"` instead of
   `"-0.5000"`. The fix computes the sign explicitly and formats the
   magnitude via `unsigned_abs()`. This silently flipped the sign of any
   correlation, 0-1 index, or regression residual that crossed the
   stringify path. `unsigned_abs()` also avoids the `i128::MIN` overflow
   panic that `.abs()` would have hit.

   The MCP `query` tool (`row_value_to_json`) serializes NUMERIC via
   `to_string()` then `parse::<f64>()`, so `CAST(-0.5 AS numeric(10,4))`
   was returning JSON `0.5` instead of `-0.5`.

2. **Node bindings: `getF64` was decoding NUMERIC as garbage / NaN.**
   This is a separate root cause from tableau#84's Display bug, fixed here for
   the same test surface. `extract_row` and `extract_chunk_columnar` were
   calling `row.get_f64(i)` for `SqlType::Numeric` columns. `get_f64`
   only handles native f32/f64 wire types; for NUMERIC it reinterpreted
   the unscaled-integer bytes as IEEE-754 → garbage values or NaN,
   regardless of sign. Switched to schema-aware `row.get_numeric(i)`,
   which honors the column scale and decodes the 8/16-byte / Arrow
   Decimal wire forms correctly. Added a `CellValue::Numeric(Numeric)`
   variant; `getString` returns the exact decimal text (preserving scale
   and sign), `getFloat64` returns the lossy-but-correct double, and
   `getInt32`/`getInt64` return the truncated integer. `getBigInt` on a
   `NUMERIC(p, 0)` column now preserves the full unscaled `i128` value
   (use it instead of `getInt64` for integer NUMERIC values above
   `Number.MAX_SAFE_INTEGER`); on `NUMERIC(p, scale>0)` it returns
   `null` rather than silently truncating. The columnar fast path
   surfaces NUMERIC as correct `f64` values.

## Test coverage

- Core: `test_numeric_display_negative_sign_preserved` — covers -0.5,
  -0.999, -0.0001, -1.5, 0, +0.5, scale-0 negative, and `i128::MIN`
  (no panic).
- MCP: `engine_negative_sub_unit_numeric_keeps_sign` — exercises the
  exact reproduction from the issue body via `execute_query_to_json`.
- Node smoke (section 18b): row-wise `getString`/`getFloat64`, AVG
  aggregate, columnar fast path, `getBigInt` precision preservation
  for `NUMERIC(38,0)` values above `Number.MAX_SAFE_INTEGER`,
  `getBigInt` returning null on `NUMERIC(p, scale>0)`.

Adversarial review (line-level + architect in parallel) caught the
missing `getBigInt` Numeric arm and the smoke-test gap; both addressed
before commit.

The Cargo.lock regen in `hyperdb-api-node/package-lock.json` removes
stale `version: "0.1.3"` and `optionalDependencies` entries that
violate the version-consistency CI guard (npm package versions are
injected at publish time, not tracked in source).

Closes tableau#84.
@StefanSteiner StefanSteiner force-pushed the fix/numeric-sign-loss-84 branch from f2af85a to a9270a7 Compare May 29, 2026 05:17
@StefanSteiner StefanSteiner merged commit d504a71 into tableau:main May 29, 2026
11 checks passed
@StefanSteiner StefanSteiner mentioned this pull request May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NUMERIC values in (-1, 0) lose their sign in Display (e.g. -0.5 renders as 0.5)

1 participant