Skip to content

perf(index): answer the FTS row-order check from a cached invalid verdict - #10553

Open
LuciferYang wants to merge 3 commits into
lance-format:mainfrom
LuciferYang:perf/fts-cached-order-verdict
Open

LuciferYang wants to merge 3 commits into
lance-format:mainfrom
LuciferYang:perf/fts-cached-order-verdict

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Closes #10551.

dense_and_strictly_ascending now returns false straight from a cached Duplicate or OutOfOrder verdict. Before, it asked try_ordered_row_addresses for the ordered view, which rescans the documents up to the first violation to rebuild the error, and then discarded the error. Callers that need the diagnostics still get them from try_ordered_row_addresses, and Unknown and Ordered verdicts take the same path as before.

On a 10M-document projection whose first violation is at document 5M, a call with a cached OutOfOrder verdict goes from 9.7 ms to 41 ns in a release build. combined_fields search makes two or three such calls per partition per query, inline on the query thread.

How was this patch tested

ascending_address_gate_requires_live_slots_and_strict_order now calls the check a second time and asserts that the validation visit counter does not move. The three cases with a duplicate or out-of-order address fail without the change.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer performance labels Nov 1, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Gate recommendation: approve.

The cached verdict is sufficient for this boolean check, and diagnostic callers keep the existing validation path. All eight ordering/liveness regression cases passed.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Nov 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer K-approved Latest Gatekeeper recommendation permits acceptance. performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: FTS row-order check rescans documents for partitions already known to be out of order

1 participant