Skip to content

HNSW: allocate incoming edge lists lazily in one compact block - #1118

Closed
dor-forer wants to merge 1 commit into
mainfrom
claude/brave-mclean-d69093
Closed

dor-forer wants to merge 1 commit into
mainfrom
claude/brave-mclean-d69093

Conversation

@dor-forer

@dor-forer dor-forer commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Describe the changes in the pull request

Every HNSW element level owned a heap allocated vecsim_stl::vector<idType> for its incoming unidirectional edges, even when the list was empty. That object costs 72 bytes (64 B object plus the 8 B allocator header) per level, and on DBpedia 100k (M=16, efC=100) 86.7% of level 0 lists are empty.

ElementLevelData now holds a pointer to a single allocation (IncomingEdges: uint32 size, uint32 capacity, then the ids), which is null while the list is empty and freed when it empties again. Growth (doubling from one), the shrink rule (capacity above twice the size) and reserve on load match the previous std::vector behavior, so capacities, the graph and the serialized format are all unchanged. Readers get a std::span<const idType>; writers pass the index allocator. Locking is unchanged: every mutation still happens under the same element locks.

Measured on DBpedia 100k x 768 FP32, IP, M=16, efC=100, single thread, same seed and insertion order:

Bytes per vector (allocator) Before After
Incoming edge lists 88.99 13.62
Whole index 3390.97 3315.60
Non-vector part 308.15 232.78
  • Fingerprints of all links and all incoming lists, the top-10 results of 500 queries, and recall@10 (0.9806) are identical before and after.
  • An index file saved before this change loads after it with identical fingerprints.

EstimateElementSize keeps its meaning (the per element cost of growing the index by a block, which the size estimation tests and RediSearch's block size computation rely on). Only its comment changed, to list what a stored vector allocates beyond it.

The new HNSWMemoryDiagnostic.breakdown test always prints the struct sizes. When VECSIM_MEM_DIAG_NPY points to a float32 .npy dataset, it also builds (or loads) an index and prints a per element memory breakdown, incoming list and link histograms, and graph and query result fingerprints. It skips in CI.

Testing: all tests of test_hnsw, test_allocator, test_hnsw_sq4, test_hnsw_sq8, test_bf16, test_fp16, test_int8, test_uint8, test_index_test_utils and test_common pass in a release build, and the same binaries are clean under ASAN, except one test_common failure that also happens without this change. When test_common runs as one process, CommonAPITest.Uint8CosineSelfDistanceAtLargeDimension crashes in test_log_impl (test_common.cpp:593), because the earlier log tests leave a logger callback installed with a destroyed context. It passes alone and does not involve HNSW.

Which issues this PR fixes

  1. Per vector memory overhead of the in-RAM HNSW index.

Main objects this PR modified

  1. ElementLevelData / ElementGraphData (graph_data.h)
  2. HNSW incoming edge writers (hnsw.h) and the serializer (hnsw_serializer_impl.h)

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

🤖 Generated with Claude Code


Note

Medium Risk
Touches core HNSW graph bookkeeping (insert, delete, repair, serialize) under existing element locks; behavior is intended to match prior vector semantics but memory layout changes are sensitive to corruption or ABI assumptions.

Overview
Replaces per-level vecsim_stl::vector incoming-edge storage with lazy, compact IncomingEdges allocations (header + id array, null when empty). ElementLevelData now exposes incoming ids via std::span, grows/shrinks/frees through VecSimAllocator, and all HNSW graph mutation paths pass the index allocator into add/remove/shrink APIs.

Serialization, integrity checks, benchmarks, and allocator unit tests are updated for the new representation; EstimateElementSize documentation clarifies what per-vector memory is outside block-growth estimates. Adds optional HNSWMemoryDiagnostic.breakdown (env-driven) for struct sizes, histograms, and per-element memory breakdown when building/loading an index from .npy data.

Reviewed by Cursor Bugbot for commit 3ab89d7. Bugbot is set up for automated code reviews on this repo. Configure here.

Every element level used to own a heap allocated vecsim_stl::vector for its
incoming unidirectional edges, even when the list was empty (87% of level 0
lists on DBpedia 100k, M=16). Each level now holds a pointer to a single
allocation (size, capacity, ids) that stays null while the list is empty.
Growth and shrink happen at the same points and to the same capacities as
before, so the graph and the serialized format are unchanged.

On DBpedia 100k x 768, M=16, efC=100 the allocator reports 75.4 bytes less
per vector (3390.97 to 3315.60), with identical links, incoming lists, top-10
query results and recall. Index files saved before this change still load.

EstimateElementSize keeps its per-block meaning; its comment now lists what a
stored vector allocates beyond it. Add a diagnostic test
(HNSWMemoryDiagnostic.breakdown) that prints the struct sizes and, given a
dataset via VECSIM_MEM_DIAG_NPY, a per-element memory breakdown and graph
fingerprints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.54%. Comparing base (f2ddf8e) to head (3ab89d7).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1118      +/-   ##
==========================================
+ Coverage   97.52%   97.54%   +0.02%     
==========================================
  Files         157      157              
  Lines       10012    10055      +43     
==========================================
+ Hits         9764     9808      +44     
+ Misses        248      247       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dor-forer

Copy link
Copy Markdown
Collaborator Author

Closing: measured on DBpedia 100k, the saving is 2.2% of index memory for FP32 (10.7% for SQ4), which does not justify the change for now.

@dor-forer dor-forer closed this Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant