Skip to content

Add TQ2/TQ3 as quantized StorageTypes - #1108

Closed
dor-forer wants to merge 1 commit into
mainfrom
codex/tq23-rabitq-draft
Closed

dor-forer wants to merge 1 commit into
mainfrom
codex/tq23-rabitq-draft

Conversation

@dor-forer

@dor-forer dor-forer commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Describe the changes in the pull request

This adds TQ2 and TQ3 (TurboQuant, 2-bit and 3-bit codes) as HNSW quantization types, using the hooks SQ8 and SQ4 already use. There is no new framework and no change to existing APIs, SQ8/SQ4 behavior or serialized bytes.

  • Codec: quantization/turboquant.h holds an MSE TurboQuant estimator for FP32 inner product, with a projection-match scale and a mean-error correction. Records are the packed codes plus 8 bytes. Queries hold the rotated query and its dot product with the mean. The rotation is a seeded dense random orthogonal matrix (seed 42 by default). TQ2/TQ3: Replace the dense rotation with a block Hadamard rotation #1124 replaces it with a block Hadamard rotation.
  • Components: quantization/turboquant_components.h adds a TQ preprocessor and a TQ calculator that share one codec. The calculator uses the existing stateful distance dispatch.
  • Training: tiered HNSW trains the TQ mean the same way it trains SQ8's. HNSWIndex::setQuantizationMean passes the mean to the TQ codec when the index is TQ, otherwise to SQ8 as before.
  • Factory: FP32 with IP, or cosine with normalized input. Anything else returns NULL, or SIZE_MAX from the estimators, the same way unsupported SQ4 configurations are rejected.
  • Serialization (test helpers only): V5 is unchanged. quantType is followed by the TQ fitted state (rotation, centroids and mean terms) instead of the SQ8 mean.

Public API: VecSimQuant_TQ2 = 3, VecSimQuant_TQ3 = 4, and VecSimTurboQuantParams {mean, seed} passed through HNSWParams.quantParams.

Compared with the previous version of this PR, this drops the storage-context layer (getQuantizationRuntime, storage_context.h, quantization_runtime.h), the V6 format, new exceptions on add/update/query, new info fields, the submit-path changes, the SVE2 kernel, the research docs and the benchmark harness. It goes from 5,002 to 1,445 added lines.

Validation (dorer-intel Xeon 8375C, and dorer-arm-big Neoverse V2, Release)

  • test_hnsw_turboquant 16/16, test_hnsw_sq8 209/209, test_hnsw_sq4 all pass, test_components 59/59, test_allocator 7/7, and test_hnsw passes on both machines.
  • The ids and scores of 200 top-10 results match the previous version of this PR exactly, for TQ2 and TQ3, with the default config and with a custom mean and seed.
  • test_common crashes at Uint8CosineSelfDistanceAtLargeDimension on main as well, so it is unrelated to this PR.
  • test_hnsw_sq4 used the raw value 3 as an "unknown quantizer". It now uses 5.

Benchmark (this commit, DBpedia 768-dim, cosine through normalized IP; 100,000 base rows and 500 held-out queries; fixture checksum verified; one pinned core, Release)

TQ2 TQ3
Exhaustive recall@10, mean (SD) of codec seeds 42, 1, 2, 3, 4 87.18% (0.36) 92.82% (0.31)
Record bytes per vector 200 296
Query prep, Xeon 8375C / Neoverse V2 641 / 61 us 641 / 61 us
Encode per vector, Xeon / Neoverse 695 / 434 us 714 / 441 us
Shared codec allocation 2.37 MB 2.37 MB
Tiered HNSW (M 16, efC 100), whole query at ef 40/80/160, Xeon 1.315 / 1.746 / 2.521 ms 1.331 / 1.758 / 2.555 ms
Same, Neoverse 0.261 / 0.381 / 0.619 ms 0.292 / 0.448 / 0.735 ms
HNSW recall@10 at ef 40/80/160, seed 42 83.90 / 86.30 / 87.48% 89.04 / 91.30 / 92.44%

Not done here

  • Python bindings.
  • Million-vector recall and latency qualification.

Which issues this PR fixes

  1. None. This is quantization research follow-up to Generalize the scalar quantization pipeline over a StorageType #1101 and the SQ4 work.

Main objects this PR modified

  1. New: src/VecSim/quantization/turboquant*.h and src/VecSim/types/turboquant.h.
  2. TQ branches in hnsw_factory.cpp, tiered_factory.cpp, hnsw.h (setQuantizationMean) and hnsw_serializer_impl.h.
  3. Tests: tests/unit/test_hnsw_turboquant.cpp.

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

🤖 Generated with Claude Code

@dor-forer dor-forer changed the title Add TQ2/TQ3 HNSW modes and optional RaBitQ1 prototype Add TQ2/TQ3 HNSW modes using shared quantization training Nov 8, 2026
@dor-forer dor-forer changed the title Add TQ2/TQ3 HNSW modes using shared quantization training Add TQ2/TQ3 as shared StorageTypes Nov 8, 2026
Base automatically changed from codex/quant-storage-type to main November 10, 2026 15:27
Add TurboQuant 2-bit and 3-bit MSE quantization for FP32 inner-product and
normalized cosine HNSW indexes, standalone and tiered.

- VecSimQuant_TQ2/TQ3 and an optional VecSimTurboQuantParams (mean, seed;
  default seed 42 and a zero mean).
- A dense-rotation codec with projection-match scaling and a mean-error
  correction. Encoding and scoring never throw; only construction and
  loading validate.
- TurboQuantPreprocessor and TurboQuantCalculator share one codec; the
  calculator returns a stateful dispatch bound to it.
- Tiered training reuses SQ8TrainingState and installs the trained mean
  through HNSWIndex::setQuantizationMean.
- Serialization V5 writes the fitted state (including the full rotation)
  after quantType. SQ8 and SQ4 bytes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dor-forer
dor-forer force-pushed the codex/tq23-rabitq-draft branch from 204ac18 to bc5ed80 Compare November 15, 2026 12:57
@dor-forer dor-forer changed the title Add TQ2/TQ3 as shared StorageTypes Add TQ2/TQ3 as quantized StorageTypes Nov 15, 2026
@dor-forer

Copy link
Copy Markdown
Collaborator Author

Moved to RediSearch/RediSearch#12352, since VecSim now lives in RediSearch under src/vecsim.

@dor-forer dor-forer closed this Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant