Skip to content

[MOD-21133] Implement MVP SQ8 for FP64 - #1121

Closed
alexd-redis wants to merge 8 commits into
mainfrom
alexd-mod-21133-mvp-sq8-fp64
Closed

alexd-redis wants to merge 8 commits into
mainfrom
alexd-mod-21133-mvp-sq8-fp64

Conversation

@alexd-redis

@alexd-redis alexd-redis commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Describe the changes in the pull request

Add reusable, untrained FP64 SQ8 components with double metadata, queries and distance results. For [1e8, 1e8 + 1] versus [1e8, 1e8], squared L2 remains 1 instead of losing the difference through float narrowing.

Share the stored-query L2 loop between SQ8 FP32 and FP64 using precision derived from the storage type; preserve arithmetic order and unaligned reads.

Scope is component support; cosine inputs must already be normalized. Training, public quantized-index admission and persistence are excluded. Disk integration belongs to [MOD-19593 Implement FLOAT64 vectors in disk HNSW](https://redislabs.atlassian.net/browse/MOD-19593); the FP64 persisted layout/version must be chosen there alongside trained-SQ8 compatibility.

Which issues this PR fixes

Main objects this PR modified

  • FP64 SQ8 storage type, preprocessor, scalar distance functions, dispatch and component factory.
  • Shared stored-query L2 helper with a compile-time guard against double-to-float wrapper misuse.
  • Component/distance tests, including exact cancellation-sensitive and unaligned cases.

Validation

  • macOS ARM64 Debug/Release: 1,626 affected tests passed each; Python SQ8/training: 127 passed.
  • Linux ARM64/GCC 12.3 Debug/Release: 1,930 affected tests passed each, 12 existing skips.
  • GCC float before/after probe: 7,200 bit-identical paired results across Debug/Release and contraction off/fast.
  • Precision test passes the real kernel and rejects private wrong-order mutations on Clang/GCC with contraction off/fast.
  • Formatting/whitespace checks passed; x86 execution is pending CI.

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Note

Medium Risk
New numerical distance/preprocessing code on the core VecSim path; behavior is precision-sensitive but isolated behind new FP64 SQ8 types and heavily tested, with no serialization/API changes in this PR.

Overview
Adds an MVP scalar-quantized (SQ8) path for double vectors with double metadata, full-precision queries, and double distance results—aimed at cases like large nearby values where FP32 narrowing would erase tiny L2 gaps.

The change introduces vecsim_types::sq8_fp64, SQ8FP64Preprocessor (storage/query/in-place quantization), scalar L2/IP kernels for stored–stored and stored–query modes, GetDistFunc dispatch in spaces.cpp, and CreateSQ8FP64IndexComponents to wire calculator + preprocessor. Cosine assumes inputs are already normalized; training, index admission, and persistence are out of scope here.

Shared L2 helper: quantized_kernels.h gains a generic StoredQueryL2 driven by each storage type’s metadata_type; existing FP32_L2Sqr delegates to it. sq8 now declares metadata_type = float explicitly.

Unit tests cover layouts, extreme magnitudes, unaligned blobs, ranking preservation (e.g. 1e8 offsets), and analytical error bounds.

Reviewed by Cursor Bugbot for commit 50fb62c. Bugbot is set up for automated code reviews on this repo. Configure here.

@CLAassistant

CLAassistant commented Nov 12, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.57%. Comparing base (f2ddf8e) to head (50fb62c).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1121      +/-   ##
==========================================
+ Coverage   97.52%   97.57%   +0.05%     
==========================================
  Files         157      160       +3     
  Lines       10012    10150     +138     
==========================================
+ Hits         9764     9904     +140     
+ Misses        248      246       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Metric == VecSimMetric_Cosine);

public:
SQ8FP64Preprocessor(std::shared_ptr<VecSimAllocator> allocator, size_t dim);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we extend QuantPreprocessor to support FP64 instead of introducing a parallel implementation? Most of the preprocessing flow is identical. The arithmetic and metadata types could follow the storage type, with direct division retained for FP64 where reciprocal overflow matters. Is there a concrete constraint that makes sharing the implementation impractical?

squares += value * value;
}
}
const double metadata[] = {minimum, delta, sum, squares};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FP64 IP and L2 kernels only consume the codes, minimum, and delta. Do we need the stored/query sums and sums of squares yet? Unless another consumer requires them, could we avoid computing and storing unused metadata?

// Cosine storage and query inputs must already be normalized by the caller.
template <VecSimMetric Metric>
IndexComponents<double, double>
CreateSQ8FP64IndexComponents(const std::shared_ptr<VecSimAllocator> &allocator, size_t dim) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we extend the existing quantized component factory and preprocessor to support FP64 instead of adding parallel implementations? Most of the setup and preprocessing flow is shared. Precision-specific arithmetic can remain separate where necessary.

// SQ8 bytes followed by double reconstruction metadata; distinct from the FP32 SQ8 layout.
struct sq8_fp64 : quantized_metadata {
using value_type = uint8_t;
using metadata_type = double;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping minimum and delta as double makes sense for general FP64 support. A constant vector [100000001, 100000001] has zero codes; storing its minimum as float rounds it to 100000000 and gives squared L2 distance 2 against the original vector instead of 0. Delta also needs double to cover ranges such as 1e100 and 1e-300, where float would overflow or underflow. I would keep these two fields as double and focus metadata savings on the unused sums discussed separately.

const double close[] = {1.0, 1.0 + std::ldexp(1.0, -40)};
const auto precise = preprocessors->preprocessForStorage(close, sizeof(close));
EXPECT_DOUBLE_EQ(readMetadata(precise.get(), 2, sq8_fp64::DELTA), std::ldexp(1.0, -40) / 255);
EXPECT_NEAR(calculator->calcDistance(precise.get(), precise.get(), 2), 0.0, 1e-40);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add a stored-to-stored precision regression with two distinct blobs sharing a large minimum but differing by small code contributions, asserting a nonzero distance in both operand orders? This assertion compares a blob with itself, while the dedicated small-residual test exercises stored-to-query L2. A distinct-blob case would protect the cancellation-resistant stored-to-stored formula against being replaced with naive reconstructed subtraction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants