Repository navigation
Generalize the scalar quantization pipeline over a StorageType - #1101
Conversation
No behavior change: SQ8 stays bit-for-bit identical. QuantPreprocessor and DistanceCalculatorWithNorm take a StorageType parameter that defaults to sq8, and sq8 gains the hooks the pipeline now goes through (max_code, data_bytes_count, load_code, store_code). CreateQuantizedIndexComponents holds the shared factory body and CreateSQ8IndexComponents wraps it. This prepares the pipeline for additional storage formats. PreprocessorsTest.QuantizedStorageSQ8BytesUnchanged checks SQ8 blobs against bytes derived with exact arithmetic, independent of the template. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the scalar SQ8-FP32 and SQ8-SQ8 inner product and L2 kernel bodies into spaces::quantized templates in quantized_kernels.h. Codes are read through StorageType::load_code and metadata is located with data_bytes_count, so a future storage type can reuse them. The SQ8 functions become thin wrappers. A new test pins bit identity against verbatim copies of the original kernels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
28b1033 to
07c2e95
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1101 +/- ##
=======================================
Coverage 97.37% 97.38%
=======================================
Files 145 147 +2
Lines 9575 9601 +26
=======================================
+ Hits 9324 9350 +26
Misses 251 251 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| using value_type = uint8_t; | ||
| static constexpr uint8_t max_code = 255; | ||
|
|
||
| static constexpr size_t data_bytes_count(size_t dim) { return dim; } |
There was a problem hiding this comment.
also const, no?
| static constexpr size_t data_bytes_count(size_t dim) { return dim; } | |
| static constexpr size_t data_bytes_count(size_t dim) const { return dim; } |
There was a problem hiding this comment.
Not changed: const is not allowed on a static member function (there is no this to qualify), so this would not compile. It is already static constexpr.
🤖 Addressed by Claude Code
|
|
||
| static constexpr size_t data_bytes_count(size_t dim) { return dim; } | ||
|
|
||
| static uint8_t load_code(const uint8_t *data, size_t index) { return data[index]; } |
There was a problem hiding this comment.
also const?
| static uint8_t load_code(const uint8_t *data, size_t index) { return data[index]; } | |
| static uint8_t load_code(const uint8_t *data, size_t index) const { return data[index]; } |
There was a problem hiding this comment.
Not changed: same as above, a static member function cannot be const. These hooks are called on the type (StorageType::load_code), never on an instance.
🤖 Addressed by Claude Code
| template <typename StorageType> | ||
| static inline float FP32_L2Sqr(const void *stored, const void *query, size_t dim) { | ||
| // Storage metadata follows a byte payload and is not necessarily float-aligned. | ||
| const auto *pVect1 = static_cast<const uint8_t *>(stored); |
There was a problem hiding this comment.
It's going to be uint8_t for any storage type?
There was a problem hiding this comment.
Yes. uint8_t here is the raw byte view of the stored blob, not the code type: data_bytes_count and the metadata offsets are in bytes, and load_code takes a byte pointer. SQ4 packs two codes per byte and reads them through the same pointer.
🤖 Addressed by Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Renames the SQ8-only factory helpers (stored size, metric resolution, parameter validation, index construction) to quantization-generic names and gives the stored-size helper a StorageType parameter (default sq8). The unknown-quantizer check moves into the shared validation. SQ8 is still the only accepted quantizer, so behavior is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the SQ8 code dot loops into sq8::code_dot and sq8::code_dot_fp32 so the generic scalar kernels call per-type hooks with unchanged arithmetic. Add a run-time CreateQuantizedIndexComponents overload that picks WithMean from mean_ptr, and make NewIndex_Quantized generic over StorageType. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit 6a60e40.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Prepares the scalar quantization pipeline for a second storage format (SQ4, which follows in #1099) without changing any behavior. SQ8 output stays bit-for-bit identical.
vecsim_types::sq8gains storage hooks:max_code,data_bytes_count(dim),load_codeandstore_code.QuantPreprocessor,DistanceCalculatorWithNormand the components factory (CreateQuantizedIndexComponents, withCreateSQ8IndexComponentskept as a wrapper) take aStorageTypetemplate parameter that defaults tosq8.SQ8_FP32_InnerProduct_Impl,SQ8_FP32_L2Sqr,SQ8_SQ8_InnerProduct_Impl,SQ8_SQ8_L2Sqr) move intospaces/quantized_kernels.hasstatic inlinetemplates. They keep the same loop order and accumulators, and the SQ8 names remain as thin wrappers. SIMD translation units must not include this header.GetQuantizedStoredDataSize<Metric, StorageType = sq8>,ResolveQuantizedMetric,QuantizationParamsSupported(quant_type, type, metric)(which now also rejects unknown quantizers) andNewIndex_Quantized. SQ8 is still the only accepted quantizer, so SQ4: 4-bit scalar quantization for standalone HNSW #1099's factory diff contains only the SQ4 additions.sq8::code_dot(stored-to-stored) andsq8::code_dot_fp32(stored-to-FP32) hold the loops that were inline in the two inner-product templates, moved unchanged, and the templates callStorageType::code_dot*. A packed type can then read each byte once.CreateQuantizedIndexComponents<DataType, Metric, StorageType>(allocator, dim, mean_ptr)picks the centered or plain components at run time for any storage type;CreateSQ8IndexComponentsdelegates to it, andNewIndex_Quantizedtakes aStorageTypeparameter.Testing
PreprocessorsTest.QuantizedStorageSQ8BytesUnchanged: full SQ8 blobs compared against independently computed bytes. It covers FP32 and FP16 input; L2, IP and Cosine; centered and plain; dims 2, 5, 37 and 128.SQ8GenericScalarKernelsBitIdentical: all four kernels compared bit for bit against verbatim copies of the originalorigin/mainbodies, over dims 1 to 768 covering everydim % 4remainder.ctest3280/3280 passed on this branch alone at2c95db85(before the code-dot hooks and the generic components wrapper;bdc5d139is checked by CI). Before them, 3278/3278 at an earlier head, includingtest_spaces1594,test_hnsw_sq8209 andtest_components59.test_spaces,test_hnsw_sq8andtest_componentspass.🤖 Generated with Claude Code
Note
Medium Risk
Touches core distance/preprocessor/index construction paths for quantized HNSW, but SQ8 output and kernels are pinned by new bit-identity tests; only SQ8 is accepted at runtime.
Overview
Refactors the HNSW scalar-quantization stack so SQ8 is one pluggable
StorageType, preparing for additional formats (e.g. SQ4) without changing SQ8 behavior—tests assert bit-identical blobs and distance kernels.Shared infrastructure: adds
quantized_metadataandspaces::quantizedkernel templates;vecsim_types::sq8inherits metadata layout and exposesmax_code, code read/write, andcode_dot*hooks.QuantPreprocessor,DistanceCalculatorWithNorm, andCreateQuantizedIndexComponentsare templated onStorageType(SQ8 wrappers remain).Factory / load path: HNSW helpers are renamed/generalized (
GetQuantizedStoredDataSize,NewIndex_Quantized,LoadIndex_Quantized,QuantizationParamsSupported);IP.cpp/L2.cppSQ8 entry points delegate to the generic kernels. Serializer wording shifts from “SQ8” to “scalar quantization”; on-disk layout is unchanged.Reviewed by Cursor Bugbot for commit af9a7da. Bugbot is set up for automated code reviews on this repo. Configure here.