Repository navigation
SQ4: 4-bit scalar quantization for standalone HNSW #1099
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
dor-forer
merged 17 commits into
codex/quant-storage-type
from
codex/sq4-production-backend
Nov 10, 2026
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
be989d4
SQ4: Add packed 4-bit scalar quantization for standalone HNSW
dor-forer 3672c75
SQ4: Expose SQ4 in the Python bindings
dor-forer f972602
SQ4: Apply review decisions
dor-forer da7f3e3
Use shared quantized snapshot loader for SQ4
dor-forer 2c6c28f
SQ4: Accumulate the centered IP correction while quantizing
dor-forer a202b78
SQ4: Skip unused L2 query reductions
dor-forer 9f3f5b9
SQ4: Reuse each packed byte in scalar FP32 L2
dor-forer 25ab7a2
Merge codex/quant-storage-type into codex/sq4-production-backend
dor-forer 5c49549
Merge codex/quant-storage-type into codex/sq4-production-backend
dor-forer d6dee30
SQ4: Compute centered IP correction from original input
dor-forer 4860441
SQ4: Multiply code_dot nibbles in 64-bit
dor-forer f09578b
Merge shared metadata extraction into SQ4
dor-forer bf9f993
Use shared quantization metadata directly for SQ4
dor-forer 117a2b4
Merge quantized code accessor rename into SQ4
dor-forer b8ad852
Rename SQ4 code accessors to get_code and set_code
dor-forer a9efe8c
Document scalar SQ4 code dot-product callers
dor-forer e01beba
Merge codex/quant-storage-type into codex/sq4-production-backend
dor-forer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why? and why not in
assign_query_metadata?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SQ8’s FP32-query L2 path also does not use these sums. Both kernels accumulate residual squares directly: we found precision loss from cancellation when expanding L2 into squared norms and a dot product. SQ8 preprocessing still computes the unused sums on this path, whereas SQ4 skips them.
The trailing slots currently remain as initialized padding, including for odd SVE query-pair tails in #1103. We opened MOD-21066 to benchmark and test skipping the unused calculations and removing the trailing fields, including any required tail-load changes, before deciding whether to remove them entirely.