Skip to content

[MOD-20697] Add a stop callback decoupled from the timeout callback - #1089

Closed
ofiryanai wants to merge 10 commits into
mainfrom
ofir-stop-callback
Closed

ofiryanai wants to merge 10 commits into
mainfrom
ofir-stop-callback

Conversation

@ofiryanai

@ofiryanai ofiryanai commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Describe the changes in the pull request

  1. Current: VecSim has one way to end a running search early: the timeout callback, which RediSearch calls per vector (brute force) or per candidate pop (graph searches), so it has to stay a few instructions. A query that crosses the memory limit in the middle of one long VecSim call (a brute-force scan, a range search, an SVS search) runs that call to the end before RediSearch can stop it.
  2. Change: A second, independent callback, VecSim_SetStopCallbackFunction(stopCallbackFunction) (typedef int (*stopCallbackFunction)(void *ctx)), gets the same VecSimQueryParams.timeoutCtx (no new query field) and is amortized inside VecSim at natural boundaries:
    • brute-force KNN, range, and both BF batch iterators' scoring: once before each storage block, through BruteForceIndex::scanVectors, which walks the blocks directly (no per-vector counter, no virtual iterator call; getVectorsIterator is removed with its last callers);
    • BF and HNSW batch iterators: once per getNextResults batch;
    • HNSW KNN bottom layer, HNSW range, HNSW batch scan and the upper-layer greedy search: once every VECSIM_STOP_CHECK_INTERVAL (64) candidate pops (local counter);
    • SVS KNN, range and batch iterator: folded into the existing cancel predicate with the same 64-call cadence (VecSimSharedStopPoller, atomic counter, sticky, since SVS may poll from several threads).
      A nonzero return takes the existing TimedOut exit of that site (no new reply code). All 13 VECSIM_TIMEOUT sites are covered; the timeout callback is untouched and called exactly as before. The tiered HNSW and SVS batch iterators now return the code of a stopped (or timed-out) later flat batch instead of retrying it.
  3. Outcome: No behaviour change for users of the library until a stop callback is set: the default is NULL and is never called. RediSearch uses it next (MOD_20241 stack) to read query memory inside a single VecSim search.

Design notes:

  • The callback pointer is a relaxed atomic read once per poll, so unsetting it while queries run never calls NULL.
  • Graph searches use VecSimStopPoller, a countdown (decrement + branch per pop) that reads the callback at each interval, so one installed mid-search is seen; VecSimSharedStopPoller checks for a callback on every call. HNSW already pays an indirect call to the timeout callback on every pop, so the countdown is the only per-pop addition (+0.2% / +0.5% instructions for HNSW KNN / range at dim 128). The callback may therefore do real work per call (RediSearch reads process memory on every call).
  • Brute force polls before every block, so any non-empty scan polls at least once (also before any work, so an already-requested stop ends it at once). A graph loop shorter than 64 pops never calls it (short upper-layer walks): best effort at 64-pop granularity by design.
  • SVS re-checks a finished search's outcome on the sticky flag (hasStopped()), not with another counted poll, so a search that completed is never discarded.

Tests (gtest):

  • BruteForceTest.stopCallbackOncePerBlock: exact cadence (23 vectors, block 5: 5 stop calls for KNN and range, +1 per batch; timeout calls unchanged = one per vector); stop at the 2nd call ends KNN/range/batch with TimedOut after exactly one block (range keeps its 5 matches); an index within one block polls once, an empty one never.
  • BruteForceMultiTest.stopCallbackOncePerBlockInBatchIterator: multi-value BF batch iterator, same cadence and abort.
  • HNSWTest.stopCallbackEveryIntervalOfCandidates, SVSTest.stopCallbackEveryIntervalOfCandidates (all SVS quantizations): KNN, range and batch iterator; results and timeout call count identical with and without a stop callback; stop calls in (0, timeouts/64 + 1]; stop at the first call ends each with TimedOut.
  • HNSWTieredIndexTest.BatchIteratorReturnsAFlatStop (single and multi-value): a stop on a later flat batch returns TimedOut; on multi-value the refill loop retried it forever before the fix.
  • NULL callback: covered by every existing test (default) and by the "without a stop callback" runs above.

Review findings (adversarial rounds, Codex), addressed or declined:

  • Codex P1: tiered batch iterators dropped a stopped flat batch's code (multi-value refill could spin): fixed, tested above.
  • Codex P2: pollers cached a missing callback for the whole search: fixed.
  • NULL call race when the callback is unset concurrently: fixed (atomic pointer, single load).
  • SVS post-search cancel() re-check could count as a poll and discard a finished result: fixed (sticky flag only).
  • ctx may be NULL for queries without params; C doc referenced a C++ constant: docs fixed.
  • Multi-value BF batch iterator untested: test added. Tiered HNSW/SVS not separately tested: their frontend (BF) and backend (HNSW/SVS) searches are the covered sites.
  • Second round (per-block walk): end-of-scan count asserts restored as one assert(count == this->count) in scanVectors; dead getVectorsIterator removed; doc pointer fixed; range/batch stop assertions tightened; small/empty index cases added.
  • Stop reported as VecSim_QueryReply_TimedOut, not a new code: kept by design; the caller tells a stop from a timeout through its own context (RediSearch maps by its recorded stop reason), and a new reply code would be an API change for every consumer.
  • Disk HNSW (RediSearchEnterprise vecsim_disk) is not in this repo: covered by its own follow-up.

Cost, measured through RediSearch (perf stat, server instructions:u and cycles:u per query, 100k dim-4 vectors, 4 interleaved reps x 200 queries; RediSearch with this callback installed vs its base without it): brute force KNN -33% instructions / -40% cycles, range -43% / -45%: the block walk (BruteForceIndex::scanVectors) replaces the per-vector virtual iterator call and its per-vector % blockSize, and polls the stop callback once per block. HNSW KNN +0.4-0.5% instructions (cycles within noise), HNSW range +1.3% instructions (the per-pop countdown; cycles +1.2% / -5.9% between modes). Full table in the RediSearch PR.

Which issues this PR fixes

  1. MOD-20697

Main objects this PR modified

  1. vec_sim.h / vec_sim_interface.h / vec_sim_common.h: VecSim_SetStopCallbackFunction, VECSIM_STOP, VecSimStopPoller, VecSimSharedStopPoller, VECSIM_STOP_CHECK_INTERVAL.
  2. Brute force, HNSW and SVS search loops and batch iterators (the 13 VECSIM_TIMEOUT sites).

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

🤖 Generated with Claude Code


Note

Medium Risk
Touches all major search paths and changes the public query reply enum from TimedOut to Stopped, so integrators must update handling while behavior of in-flight queries depends on new polling semantics.

Overview
Adds VecSim_SetStopCallbackFunction so searches can be cut short for reasons other than timeout (e.g. memory pressure), using the same timeoutCtx and polling at block boundaries (brute force), per batch (BF/HNSW batch iterators), or every VECSIM_STOP_CHECK_INTERVAL (64) steps in graph/SVS loops via VecSimStopPoller / VecSimSharedStopPoller.

Brute-force paths replace getVectorsIterator with BruteForceIndex::scanVectors, which walks storage blocks directly and checks stop before each block and timeout before each vector. Early exit now returns VecSim_QueryReply_Stopped instead of VecSim_QueryReply_TimedOut (timeout and stop share one code; partial results may remain, e.g. range queries).

Tiered HNSW/SVS batch iterators propagate a stopped flat batch instead of retrying an empty, non-depleted batch forever. Tests cover callback cadence and stop behavior across BF, HNSW, SVS, and tiered batch paths.

Reviewed by Cursor Bugbot for commit e8e34f6. Bugbot is set up for automated code reviews on this repo. Configure here.

ofiryanai and others added 5 commits October 31, 2026 20:27
VecSim_SetStopCallbackFunction installs a callback that asks a running
query to stop for reasons other than its timeout, such as memory pressure.
It gets the query's timeoutCtx and is amortized inside the searches: once
per vector block in brute force scans, once per batch in batch iterators,
and once every VECSIM_STOP_CHECK_INTERVAL candidate pops in HNSW and SVS
graph searches. A nonzero return takes the existing TimedOut exit of that
site. Without a stop callback (the default) nothing calls one, and the
timeout callback is called exactly as before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The stop callback pointer is atomic and read once per call, so unsetting
it while queries run never calls NULL. SVS re-checks the outcome of a
finished search on the sticky stop flag instead of another counted poll,
which could stop a search that already completed. Cover the multi-value
brute force batch iterator.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Brute force KNN, range and both BF batch scorers scan through
BruteForceIndex::scanVectors, which walks the storage blocks directly: the
stop callback is polled before each block and the timeout before each
vector, so the per-vector loop has no stop counter (and no virtual
iterator call).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
scanVectors asserts that the container holds the index's vector count, the
check the per-loop asserts made before. getVectorsIterator lost its last
callers. Tighten the brute force stop assertions and cover an index within
one block and an empty one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T11:52:43.423034Z e8e34f6 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0b670ea282

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/VecSim/vec_sim_interface.h Outdated
Comment on lines +331 to +333
VecSimStopPoller(void *ctx, size_t interval)
: ctx(ctx), interval(interval),
left(VecSimIndexInterface::hasStopCallback() ? interval : SIZE_MAX) {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Detect callbacks installed after poller construction

When an HNSW or SVS query starts while the global stop callback is NULL, this constructor permanently sets left to SIZE_MAX (and VecSimSharedStopPoller similarly caches an interval of zero). If another thread then installs a callback during that long-running query—explicitly allowed by the new API documentation—the query never observes it, so a newly requested memory-pressure stop is ignored. Avoid caching the absence of a callback for the poller's entire lifetime, or periodically recheck whether one has been installed.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ec73eb2: the graph pollers no longer cache a missing callback. VecSimStopPoller always counts and reads the callback at each interval; VecSimSharedStopPoller checks for a callback on every call, so one installed while a search runs is polled.

Comment on lines 190 to 191
if (VECSIM_TIMEOUT(this->getTimeoutCtx()) || VECSIM_STOP(this->getTimeoutCtx())) {
return new VecSimQueryReply(this->allocator, VecSim_QueryReply_TimedOut);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Propagate BF stop errors through tiered batch iterators

When a tiered SVS or multi-value/quantized tiered HNSW iterator is fetching its second or later batch and this new stop check fires in the frontend BF iterator, the refill loops in svs_tiered.h:403-408 and hnsw_tiered.h:1268-1273 discard tail->code. A persistent memory-pressure callback therefore makes those loops repeatedly request an empty, timed-out batch without advancing or becoming depleted, hanging instead of returning VecSim_QueryReply_TimedOut; even a one-shot callback is silently ignored. The tiered refill paths need to propagate the frontend reply code before continuing.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ec73eb2: the tiered HNSW and SVS batch iterators now return the code of a later flat batch instead of retrying an empty, non-depleted one (this also applied to a persistent timeout before). Covered by HNSWTieredIndexTest.BatchIteratorReturnsAFlatStop (single and multi-value).

The tiered HNSW and SVS batch iterators dropped the code of a later flat
batch, so a stopped (or timed-out) flat iterator, empty and not depleted,
was asked again and again by the multi-value refill loop. They now return
that code. The graph pollers no longer cache a missing stop callback for
the whole search, so a callback installed while it runs is polled too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.36364% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.21%. Comparing base (001f0b0) to head (e8e34f6).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/VecSim/algorithms/svs/svs_tiered.h 25.00% 3 Missing ⚠️
src/VecSim/algorithms/svs/svs.h 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1089      +/-   ##
==========================================
- Coverage   97.39%   97.21%   -0.19%     
==========================================
  Files         145      145              
  Lines        9575     9619      +44     
==========================================
+ Hits         9326     9351      +25     
- Misses        249      268      +19     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ofiryanai ofiryanai changed the title [MOD-20241] Add a stop callback decoupled from the timeout callback [MOD-20697] Add a stop callback decoupled from the timeout callback Nov 2, 2026
ofiryanai and others added 3 commits November 2, 2026 19:26
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5af413841c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +28 to +34
bool completed = this->index->scanVectors(
this->getTimeoutCtx(), [&](const char *vector, idType curr_id) {
auto score = this->index->calcDistance(vector, this->getQueryBlob());
this->scores.emplace_back(score, this->index->getVectorLabel(curr_id));
});
if (!completed) {
return VecSim_QueryReply_TimedOut;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear partial scores after an interrupted initial scan

When the stop condition fires after the first block of a single-value BF batch iterator, scanVectors leaves the already appended entries in scores before returning TimedOut. If the condition is transient and the caller retries the same iterator—including a tiered iterator whose initial flat scan was stopped—getNextResults sees that scores is nonempty, skips calculateScores, and returns results from only the scanned prefix while permanently omitting the remaining labels. Clear the partial scores on failure or track successful score initialization separately.

Useful? React with 👍 / 👎.

Comment on lines +53 to 56
if (VECSIM_TIMEOUT(timeoutCtx) || stop.hasStopped()) {
rep->code = VecSim_QueryReply_TimedOut;
rep->results.clear();
return rep;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve consumed SVS results when a refill is stopped

When a batch request first consumes the remaining entries in the current SVS cache and then the new stop callback fires inside impl_.next, this branch clears the reply even though curr_it has already advanced past those cached entries. If the stop is transient and the caller retries the iterator, those cleared neighbors are no longer available and are silently omitted from subsequent batches. Buffer or restore the consumed entries when the refill returns TimedOut so retrying does not lose results.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5af4138. Configure here.

auto timeoutCtx = this->getTimeoutCtx();
auto cancel = [timeoutCtx]() { return VECSIM_TIMEOUT(timeoutCtx); };
VecSimSharedStopPoller stop(timeoutCtx, VECSIM_STOP_CHECK_INTERVAL);
auto cancel = [timeoutCtx, &stop]() { return VECSIM_TIMEOUT(timeoutCtx) || stop.due(); };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SVS batch ignores pending stop

Medium Severity

Each getNextResultsImpl constructs a new VecSimSharedStopPoller, so the opening cancel() is only the first due() of that object and does not invoke the stop callback. When leftover SVS hits are served from cache, impl_.next() is skipped and a requested stop is ignored, unlike timeout at this site and unlike the BF/HNSW once-per-batch VECSIM_STOP checks. Hybrid queries that ask for fewer results than the search window commonly take this cache path, so a memory-pressure stop can still return another batch.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5af4138. Configure here.

The stop callback ends a query early with the same code as the timeout
callback, and VecSim cannot tell which one fired. Name the code for what
it means: ended early on a timeout or a requested stop, with partial
results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e8e34f6002

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +108 to +109
if (VECSIM_TIMEOUT(this->getTimeoutCtx()) || stop.due()) {
return VecSim_QueryReply_Stopped;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve HNSW scan state when stopping a batch

When a transient stop fires from this poller after the scan has processed candidates, those candidates have already been removed from candidates and marked visited, while their results exist only in the local top_candidates heap. The error path deletes that heap, and scanGraph may also mark the iterator depleted because the partial heap contains fewer than ef entries; after the callback clears, HasNext can therefore be false or a direct retry resumes without the discarded neighbors. Retain or roll back the in-progress scan state, and do not mark the iterator depleted when the scan was stopped.

Useful? React with 👍 / 👎.

@ofiryanai

Copy link
Copy Markdown
Collaborator Author

Replaced by RediSearch/RediSearch#12379, which carries this PR as one commit (Query Memory Guardrails Phase 1 rebuild).

@ofiryanai ofiryanai closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant