Repository navigation
SQ4: 4-bit scalar quantization for standalone HNSW - #1099
Conversation
36aa7d7 to
d1499fb
Compare
d1499fb to
0bf8bdb
Compare
168c7e6 to
971d296
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## codex/quant-storage-type #1099 +/- ##
============================================================
+ Coverage 97.38% 97.46% +0.08%
============================================================
Files 147 148 +1
Lines 9601 9715 +114
============================================================
+ Hits 9350 9469 +119
+ Misses 251 246 -5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 971d296. Configure here.
403c48c to
3ffff81
Compare
3ffff81 to
979efab
Compare
Adds VecSimQuant_SQ4: FP32 input, IP and L2 (Cosine on pre-normalized vectors resolves to IP), optional mean centering, standalone HNSW only. Codes are packed two per byte, followed by the SQ8 metadata. Distances use the generic scalar kernels from the StorageType split; the FP32-query IP falls back to per-element reconstruction on a non-finite result. Unsupported configurations are rejected with NULL, and the existing V5 serializer stores the quant type and mean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build SQ4 components through the generic CreateQuantizedIndexComponents wrapper and drop the SQ4-specific factory overloads. Remove the SQ4-FP32 inner product overflow fallback so SQ4 returns the plain distance like SQ8. Give sq4 packed code dot hooks that read each byte once, and name the nibble shift bits_per_code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
979efab to
f972602
Compare
The SQ4 mean correction x_mean_ip is summed from each code as it is generated, in coordinate order with one double accumulator, instead of in a second pass that re-reads and decodes the packed codes. The stored bytes and the correction are unchanged. For SQ4 the input-side sum in find_min_max, which was overwritten, is skipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep two initialized FP32 padding slots for SIMD tails and update the query metadata contract and component checks. Validation on dorer-intel: SQ4 81 passed, 12 expected skips; SQ8 209 passed.
Decode both nibbles from one load, preserve the residual and coordinate accumulation order, and handle odd tails explicitly. Verify bitwise agreement with the previous scalar arithmetic across odd/even dimensions, nonzero padding, and large offsets. Validation on dorer-intel: SQ4 82 passed with 12 expected skips; SQ8 209 passed. GCC 13.3 production assembly has one byte load per pair.
Adapt SQ4 to the storage context components: the SQ4 create path passes QuantizationInit, the initial-size estimate uses the SQ4 storage type, sq4 reports its own quantization_type, and tiered creation keeps rejecting SQ4 now that the generic quantization check is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The base reverted the storage context redesign, so this restores the SQ4 code to its form on the original components API (the tree of 9f3f5b9) and drops the adaptations from the previous merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The products are at most 225 and never overflowed, but multiplying in the 64-bit accumulator type keeps static analysis from flagging a widening after a 32-bit multiply. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| // padding for SIMD tails that load a final query pair when dim is odd. | ||
| const MetadataType padding[2] = {}; | ||
| memcpy(metadata_dst, padding, sizeof(padding)); | ||
| } else { |
There was a problem hiding this comment.
why? and why not in assign_query_metadata?
There was a problem hiding this comment.
SQ8’s FP32-query L2 path also does not use these sums. Both kernels accumulate residual squares directly: we found precision loss from cancellation when expanding L2 into squared norms and a dot product. SQ8 preprocessing still computes the unused sums on this path, whereas SQ4 skips them.
The trailing slots currently remain as initialized padding, including for odd SVE query-pair tails in #1103. We opened MOD-21066 to benchmark and test skipping the unused calculations and removing the trailing fields, including any required tail-load changes, before deciding whether to remove them entirely.
| namespace vecsim_types { | ||
|
|
||
| // Two scalar-quantized coordinates per byte, followed by the same metadata as SQ8. | ||
| struct sq4 : sq8 { |
|
|
||
| static constexpr size_t data_bytes_count(size_t dim) { return dim / 2 + dim % 2; } | ||
|
|
||
| static uint8_t load_code(const uint8_t *data, size_t index) { |
There was a problem hiding this comment.
maybe get()? consider a clearer name
|
|
||
| // Dot product of two packed code arrays: sum(q1_i * q2_i). Each byte is loaded once and both | ||
| // nibbles are used. The 64-bit integer sum is exact for any dimension. | ||
| static float code_dot(const uint8_t *a, const uint8_t *b, size_t dim) { |
There was a problem hiding this comment.
consider mentioning in the comment that only naive uses this?
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Summary
Adds
VecSimQuant_SQ4 = 2, a packed 4-bit scalar quantizer for standalone HNSW. Stacked on #1101 (theStorageTyperefactor of the SQ8 pipeline); retarget tomainafter #1101 merges. SQ8 behavior is unchanged.Distances in this PR are scalar only. The ARM and x86 SIMD kernels are in the follow-up PR stacked on this one, #1103, so this PR can be reviewed for the feature itself: storage layout, validation, recall and C API safety.
On a 10k DBpedia D768 sample, centered SQ4 has 93.6% exhaustive recall@10 (SQ8: 99.4%) at 400 bytes per vector (SQ8: 784).
What is supported
HNSWParams.quantParamsVecSimQuant_SQ4,quantization_meanacceptedThe following are rejected, with NULL from
NewIndex,SIZE_MAXfromEstimateInitialSize, and no exception crossing the C API:Storage and distances
Storage is
ceil(dim/2)code bytes (even coordinate in the low nibble, padding nibble zero), then the SQ8 metadata as unaligned FP32:min, delta, sum, thensum_sqfor L2, thenx_mean_ipfor centered IP.vecsim_types::sq4(types/sq4.h) overrides theStorageTypehooks from Generalize the scalar quantization pipeline over a StorageType #1101:max_code,data_bytes_count,load_code,store_code, and the code-dot loopscode_dotandcode_dot_fp32, which read each packed byte once and use both nibbles (the stored-to-stored sum is exact in 64-bit integers). The scalar kernels are the generic templates from Generalize the scalar quantization pipeline over a StorageType #1101 instantiated withsq4.FP32-query IP is
min*y_sum + delta*sum(code*y), the same identity as SQ8, with the same behavior for extreme query values (no SQ4-specific overflow fallback).FP32-query L2 subtracts the offsets first, as SQ8 does.
SQ4-SQ4 IP uses SQ8's stored-sum identity.
SQ4-SQ4 L2 uses SQ8's stored-sum identity
||x||^2 + ||y||^2 - 2*IP(x, y), for centered and plain indexes alike.Known limitation: on un-normalized data with a large common offset, plain (uncentered) SQ4 L2 loses up to 9.6 pp HNSW recall@100 with the identity compared with per-element reconstruction; centered SQ4 is unaffected. Use a mean for such L2 data. Tracked in MOD-20999.
Every SQ4 distance chooser returns alignment 0: the packed storage and its metadata have no alignment requirement.
Components are built with Generalize the scalar quantization pipeline over a StorageType #1101's generic
CreateQuantizedIndexComponents<..., sq4>, on both the create and the load path.Commits
SQ4: Add packed 4-bit scalar quantization for standalone HNSW: the type, scalar distances, factory, preprocessing, serializer, tiered rejection and C++ tests.SQ4: Expose SQ4 in the Python bindings: the enum,quantization_mean, and the flow test.SQ4: Apply review decisions: generic components builder, no IP overflow fallback, packed code-dot loops and named nibble constants.Testing
basic testsat2c6c28f6: full unitctest3373/3373 passed (the Python step after it failed on a GitHub 504 while downloading SVS). The CI sanitizer job passed at the earlier headf9726022.ctest3376/3376 at3ffff81b, before the single L2 formula and the review decisions.🤖 Generated with Claude Code
Note
Medium Risk
Touches HNSW index construction, serialization, and distance kernels—a correctness-sensitive path—but scope is gated by validation, tiered SQ4 is blocked, and coverage is heavy in new unit/flow tests.
Overview
Adds
VecSimQuant_SQ4and wires packed 4-bit scalar quantization through standalone HNSW: create, load/save (V5quantType), memory estimation, and Python (VecSimQuant_SQ4,quantization_mean).vecsim_types::sq4packs two coordinates per byte with the same FP32 metadata layout as SQ8. Scalar L2/IP kernels cover SQ4↔FP32 (L2 uses per-residual accumulation) and SQ4↔SQ4 (generic quantized templates). Distance dispatch sets alignment 0 for packed blobs.QuantPreprocessorhandles SQ4 packing; SQ4 L2 query blobs keep zeroed trailing FP32 slots instead of query sums.Validation / limits: FP32 input and IP/L2 only (Cosine when pre-normalized → IP); tiered SQ4 is rejected; unknown quant types and bad snapshots fail without exceptions across the C API. SQ8 behavior is unchanged.
Reviewed by Cursor Bugbot for commit e01beba. Bugbot is set up for automated code reviews on this repo. Configure here.