[ABA-15] fix(vortex-turboquant): use 6-σ centroid init to prevent 2-bit collapse at padded_dim=2048#7
Open
abnobdoss wants to merge 2 commits into
Open
[ABA-15] fix(vortex-turboquant): use 6-σ centroid init to prevent 2-bit collapse at padded_dim=2048#7abnobdoss wants to merge 2 commits into
abnobdoss wants to merge 2 commits into
Conversation
…at padded_dim=2048 Adds a regression test that catches the uniform [-1,1] centroid initialisation bug described in vortex#7245 / ABA-15. At padded_dim=2048, bit_width=2, the marginal pdf (1 - x²)^((d-3)/2) is so sharply peaked near zero that outer Voronoi cells have a numerical integral of ~0. The denominator < 1e-30 guard returns the cell midpoint, outer centroids get stuck near ±0.75, and the quantizer silently degrades to 1-bit fidelity. The test asserts max|centroid| < 0.5 — which fails on the buggy init and passes after the 6-σ fix. Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
…llapse Fixes ABA-15 / vortex#7245. Replace uniform [-1, 1] centroid initialization with a bounded [-W, W] initialization where W = min(1, 6/sqrt(d)). At high padded dimensions (d >= 2048), the marginal pdf (1 - x^2)^((d-3)/2) is sharply concentrated near zero with effective std-dev sigma = 1/sqrt(d). A uniform [-1, 1] init places centroids far outside the pdf support, causing the trapezoidal integrator to return denominator ≈ 0 for outer Voronoi cells. The 1e-30 guard then returns the cell midpoint, outer centroids never converge, and bit_width=2 silently degrades to 1-bit fidelity. The 6-sigma bound ensures all initial centroids fall within the numerically representable pdf support, allowing Max-Lloyd to converge to distinct centroids for all bit widths at any valid dimension. Pre-verified against the vortex-turboquant test suite at /tmp/tq-72/. Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
padded_dim >= 2048,bit_width=2, the Max-Lloyd centroid initializer places starting points at[-0.75, -0.25, 0.25, 0.75](uniform on[-1,1]). The marginal pdf(1 - x²)^((d-3)/2)underflows to ~0 outside|x| ≈ 0.1at these dimensions. The trapezoidal integrator returnsdenominator < 1e-30for outer Voronoi cells; the guard returns the cell midpoint; outer centroids never converge. Result: 2-bit quantizer silently degrades to 1-bit fidelity.[-1, 1]init with bounded[-W, W]whereW = min(1, 6/sqrt(d)). Initializes all centroids inside the numerically representable pdf support.Linear
Closes ABA-15
Validation
issue_aba15_centroids_dont_collapse_at_padded_dim_2048_bit_width_2— assertsmax|centroid| < 0.5for(dim=2048, bit_width=2).max|c| = 0.6725), PASS after fix.vortex-turboquantunit tests pass post-fix./tmp/tq-72/vortex-turboquant-fix/— the fix patch applied cleanly with no divergence from develop.cargo fmt(stable) applied.🤖 Generated with Claude Code