Skip to content

MOD-21005: Compute centered SQ8 IP correction from reconstructed values - #1109

Closed
dor-forer wants to merge 1 commit into
codex/sq4-production-backendfrom
codex/mod-21005-sq8-reconstructed-mean
Closed

dor-forer wants to merge 1 commit into
codex/sq4-production-backendfrom
codex/mod-21005-sq8-reconstructed-mean

Conversation

@dor-forer

@dor-forer dor-forer commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Centered SQ8 inner-product blobs currently store x_mean_ip from the original input, while the stored/stored kernel operates on the reconstructed quantized vector. This mixes two representations in the mean correction. Compute the correction from min + delta * code + mean instead, using the same fused quantization path as SQ4.

This is a separate behavior change for MOD-21005, stacked on #1099. #1101 keeps its promise of preserving SQ8 behavior. Retarget after the lower PRs merge.

Changes:

  • Use reconstructed-code mean-IP accumulation for both SQ8 and SQ4, in coordinate order with a double accumulator.
  • Remove the obsolete input-side mean-IP reduction from find_min_max.
  • Update the independent SQ8 metadata expectations and add a lossy FP32/FP16 regression covering all four loop remainders, stored/stored scores, and complete in-place blob equality.
  • Plain SQ8, L2, query preprocessing, and SQ4 retain their behavior.

Compatibility: no field-layout or encoding-version change. Existing V5 snapshots retain their stored corrections when loaded; newly inserted or replaced vectors use the corrected formula. Loading does not migrate or rewrite existing vectors.

Validation on dorer-intel (GCC 13, Release), final commit e9311eb6:

  • New regression fails before the fix (about 0.19 score error for the seven-coordinate case), passes after it.
  • test_components: 60 passed.
  • test_hnsw_sq8: 209 passed.
  • test_hnsw_sq4: 82 passed; 12 intentional single-value instances of multi-only cases skipped.
  • Independent source review: no actionable findings.
  • A real pre-fix V5 snapshot with 8,192 vectors loads with identical labels and scores for 128 top-100 queries; insertion and querying a new vector also pass.

Matched DBpedia screen: 8,192 candidates (rows 0–8191), 128 queries (rows 8192–8319), D768, normalized FP32 input, mean computed from candidates, double-precision exhaustive ground truth. HNSW: M16, efConstruction200, efRuntime200, sequential insertion.

Recall Before (9f3f5b92) After (e9311eb6)
Exhaustive @10 99.6094% 99.6094%
Exhaustive @100 99.7031% 99.7031%
HNSW @10 99.6094% 99.6094%
HNSW @100 99.5469% 99.5547%

The HNSW @100 difference is one additional hit across 12,800 expected hits; this small sample establishes no material recall change, rather than a general recall improvement. Fixture: dbpedia-cosine-dim768-test_vectors.raw, SHA256 4c63a75134c43b4eefe599800c6d7a041b503cdeb1344567b7be4c28a520f2a6.

@dor-forer dor-forer closed this Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant