Skip to content

Generalize the scalar quantization pipeline over a StorageType - #1101

Merged
dor-forer merged 11 commits into
mainfrom
codex/quant-storage-type
Nov 10, 2026
Merged

dor-forer merged 11 commits into
mainfrom
codex/quant-storage-type

Conversation

@dor-forer

@dor-forer dor-forer commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Prepares the scalar quantization pipeline for a second storage format (SQ4, which follows in #1099) without changing any behavior. SQ8 output stays bit-for-bit identical.

  • vecsim_types::sq8 gains storage hooks: max_code, data_bytes_count(dim), load_code and store_code.
  • QuantPreprocessor, DistanceCalculatorWithNorm and the components factory (CreateQuantizedIndexComponents, with CreateSQ8IndexComponents kept as a wrapper) take a StorageType template parameter that defaults to sq8.
  • The four SQ8 scalar kernels (SQ8_FP32_InnerProduct_Impl, SQ8_FP32_L2Sqr, SQ8_SQ8_InnerProduct_Impl, SQ8_SQ8_L2Sqr) move into spaces/quantized_kernels.h as static inline templates. They keep the same loop order and accumulators, and the SQ8 names remain as thin wrappers. SIMD translation units must not include this header.
  • The HNSW factory helpers lose their SQ8-only names: GetQuantizedStoredDataSize<Metric, StorageType = sq8>, ResolveQuantizedMetric, QuantizationParamsSupported(quant_type, type, metric) (which now also rejects unknown quantizers) and NewIndex_Quantized. SQ8 is still the only accepted quantizer, so SQ4: 4-bit scalar quantization for standalone HNSW #1099's factory diff contains only the SQ4 additions.
  • Storage types own their code-dot loops: sq8::code_dot (stored-to-stored) and sq8::code_dot_fp32 (stored-to-FP32) hold the loops that were inline in the two inner-product templates, moved unchanged, and the templates call StorageType::code_dot*. A packed type can then read each byte once.
  • CreateQuantizedIndexComponents<DataType, Metric, StorageType>(allocator, dim, mean_ptr) picks the centered or plain components at run time for any storage type; CreateSQ8IndexComponents delegates to it, and NewIndex_Quantized takes a StorageType parameter.

Testing

  • PreprocessorsTest.QuantizedStorageSQ8BytesUnchanged: full SQ8 blobs compared against independently computed bytes. It covers FP32 and FP16 input; L2, IP and Cosine; centered and plain; dims 2, 5, 37 and 128.
  • SQ8GenericScalarKernelsBitIdentical: all four kernels compared bit for bit against verbatim copies of the original origin/main bodies, over dims 1 to 768 covering every dim % 4 remainder.
  • x86 (dorer-intel, Xeon 8375C, AVX-512): full ctest 3280/3280 passed on this branch alone at 2c95db85 (before the code-dot hooks and the generic components wrapper; bdc5d139 is checked by CI). Before them, 3278/3278 at an earlier head, including test_spaces 1594, test_hnsw_sq8 209 and test_components 59.
  • ARM (Neoverse-V2): test_spaces, test_hnsw_sq8 and test_components pass.
  • clang-format clean.

🤖 Generated with Claude Code


Note

Medium Risk
Touches core distance/preprocessor/index construction paths for quantized HNSW, but SQ8 output and kernels are pinned by new bit-identity tests; only SQ8 is accepted at runtime.

Overview
Refactors the HNSW scalar-quantization stack so SQ8 is one pluggable StorageType, preparing for additional formats (e.g. SQ4) without changing SQ8 behavior—tests assert bit-identical blobs and distance kernels.

Shared infrastructure: adds quantized_metadata and spaces::quantized kernel templates; vecsim_types::sq8 inherits metadata layout and exposes max_code, code read/write, and code_dot* hooks. QuantPreprocessor, DistanceCalculatorWithNorm, and CreateQuantizedIndexComponents are templated on StorageType (SQ8 wrappers remain).

Factory / load path: HNSW helpers are renamed/generalized (GetQuantizedStoredDataSize, NewIndex_Quantized, LoadIndex_Quantized, QuantizationParamsSupported); IP.cpp / L2.cpp SQ8 entry points delegate to the generic kernels. Serializer wording shifts from “SQ8” to “scalar quantization”; on-disk layout is unchanged.

Reviewed by Cursor Bugbot for commit af9a7da. Bugbot is set up for automated code reviews on this repo. Configure here.

@dor-forer
dor-forer added this pull request to stack #1102 November 5, 2026 08:12
@dor-forer
dor-forer requested a review from GuyAv46 November 5, 2026 08:12
@dor-forer
dor-forer marked this pull request as ready for review November 5, 2026 08:48
dor-forer and others added 2 commits November 5, 2026 11:11
No behavior change: SQ8 stays bit-for-bit identical. QuantPreprocessor and
DistanceCalculatorWithNorm take a StorageType parameter that defaults to
sq8, and sq8 gains the hooks the pipeline now goes through (max_code,
data_bytes_count, load_code, store_code). CreateQuantizedIndexComponents
holds the shared factory body and CreateSQ8IndexComponents wraps it. This
prepares the pipeline for additional storage formats.

PreprocessorsTest.QuantizedStorageSQ8BytesUnchanged checks SQ8 blobs
against bytes derived with exact arithmetic, independent of the template.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the scalar SQ8-FP32 and SQ8-SQ8 inner product and L2 kernel bodies into
spaces::quantized templates in quantized_kernels.h. Codes are read through
StorageType::load_code and metadata is located with data_bytes_count, so a
future storage type can reuse them. The SQ8 functions become thin wrappers.

A new test pins bit identity against verbatim copies of the original kernels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dor-forer
dor-forer force-pushed the codex/quant-storage-type branch from 28b1033 to 07c2e95 Compare November 5, 2026 09:11
@codecov

codecov Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.74214% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.38%. Comparing base (4d9e4d2) to head (af9a7da).

Files with missing lines Patch % Lines
src/VecSim/index_factories/hnsw_factory.cpp 95.55% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1101   +/-   ##
=======================================
  Coverage   97.37%   97.38%           
=======================================
  Files         145      147    +2     
  Lines        9575     9601   +26     
=======================================
+ Hits         9324     9350   +26     
  Misses        251      251           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/VecSim/types/sq8.h
using value_type = uint8_t;
static constexpr uint8_t max_code = 255;

static constexpr size_t data_bytes_count(size_t dim) { return dim; }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also const, no?

Suggested change
static constexpr size_t data_bytes_count(size_t dim) { return dim; }
static constexpr size_t data_bytes_count(size_t dim) const { return dim; }

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed: const is not allowed on a static member function (there is no this to qualify), so this would not compile. It is already static constexpr.

🤖 Addressed by Claude Code

Comment thread src/VecSim/types/sq8.h Resolved
Comment thread src/VecSim/types/sq8.h Outdated

static constexpr size_t data_bytes_count(size_t dim) { return dim; }

static uint8_t load_code(const uint8_t *data, size_t index) { return data[index]; }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also const?

Suggested change
static uint8_t load_code(const uint8_t *data, size_t index) { return data[index]; }
static uint8_t load_code(const uint8_t *data, size_t index) const { return data[index]; }

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed: same as above, a static member function cannot be const. These hooks are called on the type (StorageType::load_code), never on an instance.

🤖 Addressed by Claude Code

template <typename StorageType>
static inline float FP32_L2Sqr(const void *stored, const void *query, size_t dim) {
// Storage metadata follows a byte payload and is not necessarily float-aligned.
const auto *pVect1 = static_cast<const uint8_t *>(stored);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's going to be uint8_t for any storage type?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. uint8_t here is the raw byte view of the stored blob, not the code type: data_bytes_count and the metadata offsets are in bytes, and load_code takes a byte pointer. SQ4 packs two codes per byte and reads them through the same pointer.

🤖 Addressed by Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dor-forer
dor-forer requested a review from GuyAv46 November 8, 2026 07:13
Renames the SQ8-only factory helpers (stored size, metric resolution,
parameter validation, index construction) to quantization-generic names
and gives the stored-size helper a StorageType parameter (default sq8).
The unknown-quantizer check moves into the shared validation. SQ8 is
still the only accepted quantizer, so behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dor-forer and others added 2 commits November 8, 2026 10:41
Move the SQ8 code dot loops into sq8::code_dot and sq8::code_dot_fp32 so
the generic scalar kernels call per-type hooks with unchanged arithmetic.
Add a run-time CreateQuantizedIndexComponents overload that picks WithMean
from mean_ptr, and make NewIndex_Quantized generic over StorageType.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dor-forer dor-forer mentioned this pull request Nov 8, 2026
2 tasks done
@dor-forer dor-forer changed the title Generalize the scalar quantization pipeline over a StorageType Generalize quantized storage over StorageType contexts Nov 8, 2026
@dor-forer dor-forer changed the title Generalize quantized storage over StorageType contexts Generalize the scalar quantization pipeline over a StorageType Nov 8, 2026
GuyAv46
GuyAv46 previously approved these changes Nov 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dor-forer
dor-forer added this pull request to the merge queue Nov 10, 2026
Merged via the queue into main with commit 02d8c03 Nov 10, 2026
16 checks passed
@dor-forer
dor-forer deleted the codex/quant-storage-type branch November 10, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants