Repository navigation
Conversation
Every element level used to own a heap allocated vecsim_stl::vector for its incoming unidirectional edges, even when the list was empty (87% of level 0 lists on DBpedia 100k, M=16). Each level now holds a pointer to a single allocation (size, capacity, ids) that stays null while the list is empty. Growth and shrink happen at the same points and to the same capacities as before, so the graph and the serialized format are unchanged. On DBpedia 100k x 768, M=16, efC=100 the allocator reports 75.4 bytes less per vector (3390.97 to 3315.60), with identical links, incoming lists, top-10 query results and recall. Index files saved before this change still load. EstimateElementSize keeps its per-block meaning; its comment now lists what a stored vector allocates beyond it. Add a diagnostic test (HNSWMemoryDiagnostic.breakdown) that prints the struct sizes and, given a dataset via VECSIM_MEM_DIAG_NPY, a per-element memory breakdown and graph fingerprints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1118 +/- ##
==========================================
+ Coverage 97.52% 97.54% +0.02%
==========================================
Files 157 157
Lines 10012 10055 +43
==========================================
+ Hits 9764 9808 +44
+ Misses 248 247 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collaborator
Author
|
Closing: measured on DBpedia 100k, the saving is 2.2% of index memory for FP32 (10.7% for SQ4), which does not justify the change for now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the changes in the pull request
Every HNSW element level owned a heap allocated
vecsim_stl::vector<idType>for its incoming unidirectional edges, even when the list was empty. That object costs 72 bytes (64 B object plus the 8 B allocator header) per level, and on DBpedia 100k (M=16, efC=100) 86.7% of level 0 lists are empty.ElementLevelDatanow holds a pointer to a single allocation (IncomingEdges:uint32size,uint32capacity, then the ids), which is null while the list is empty and freed when it empties again. Growth (doubling from one), the shrink rule (capacity above twice the size) andreserveon load match the previousstd::vectorbehavior, so capacities, the graph and the serialized format are all unchanged. Readers get astd::span<const idType>; writers pass the index allocator. Locking is unchanged: every mutation still happens under the same element locks.Measured on DBpedia 100k x 768 FP32, IP, M=16, efC=100, single thread, same seed and insertion order:
EstimateElementSizekeeps its meaning (the per element cost of growing the index by a block, which the size estimation tests and RediSearch's block size computation rely on). Only its comment changed, to list what a stored vector allocates beyond it.The new
HNSWMemoryDiagnostic.breakdowntest always prints the struct sizes. WhenVECSIM_MEM_DIAG_NPYpoints to a float32 .npy dataset, it also builds (or loads) an index and prints a per element memory breakdown, incoming list and link histograms, and graph and query result fingerprints. It skips in CI.Testing: all tests of test_hnsw, test_allocator, test_hnsw_sq4, test_hnsw_sq8, test_bf16, test_fp16, test_int8, test_uint8, test_index_test_utils and test_common pass in a release build, and the same binaries are clean under ASAN, except one test_common failure that also happens without this change. When test_common runs as one process,
CommonAPITest.Uint8CosineSelfDistanceAtLargeDimensioncrashes intest_log_impl(test_common.cpp:593), because the earlier log tests leave a logger callback installed with a destroyed context. It passes alone and does not involve HNSW.Which issues this PR fixes
Main objects this PR modified
ElementLevelData/ElementGraphData(graph_data.h)Mark if applicable
🤖 Generated with Claude Code
Note
Medium Risk
Touches core HNSW graph bookkeeping (insert, delete, repair, serialize) under existing element locks; behavior is intended to match prior vector semantics but memory layout changes are sensitive to corruption or ABI assumptions.
Overview
Replaces per-level
vecsim_stl::vectorincoming-edge storage with lazy, compactIncomingEdgesallocations (header + id array, null when empty).ElementLevelDatanow exposes incoming ids viastd::span, grows/shrinks/frees throughVecSimAllocator, and all HNSW graph mutation paths pass the index allocator into add/remove/shrink APIs.Serialization, integrity checks, benchmarks, and allocator unit tests are updated for the new representation;
EstimateElementSizedocumentation clarifies what per-vector memory is outside block-growth estimates. Adds optionalHNSWMemoryDiagnostic.breakdown(env-driven) for struct sizes, histograms, and per-element memory breakdown when building/loading an index from.npydata.Reviewed by Cursor Bugbot for commit 3ab89d7. Bugbot is set up for automated code reviews on this repo. Configure here.