Skip to content

fix: protect fragment reuse history on stable-row-ID tables (re-land) - #10441

Open
wjones127 wants to merge 2 commits into
mainfrom
reland-9769-without-suffix-relabels
Open

wjones127 wants to merge 2 commits into
mainfrom
reland-9769-without-suffix-relabels

Conversation

@wjones127

Copy link
Copy Markdown
Contributor

Summary

Re-lands #9769 (fragment reuse history protection on stable-row-ID tables), which #10440 reverted together with #8400, adapted to compaction without #8400's suffix relabels.

The first commit is #9769 as merged. The second adapts it:

  • History validation drops the metadata-only relabel case. Compaction only rewrites the fragments it selected, so a recorded group always records the live rows of its sources.
  • Tests that relied on relabels now expect the behavior without them. Recorded and unrecorded compactions of disjoint fragments both land without a conflict, and an index keeps covering fragments compaction didn't touch. The test of relabels with deletions is removed.

Evidence

The dataset::optimize, io::commit, index::frag_reuse and dataset::index::frag_reuse unit tests pass (1139). Before the adaptation, 6 of them failed, all relabel-dependent.

Merge Risk

Door: two-way. No format change. It restores the commit-time history checks from #9769.
Blast Radius: concurrency and compaction on stable-row-ID tables with deferred index remap.

🤖 Generated with Claude Code

brendanclement and others added 2 commits October 28, 2026 22:31
Record metadata-only suffix relabels with all moved physical row addresses, including deleted rows. Integrate strict trailing-suffix planning with recorded and unrecorded compaction races, and cover the resulting Rust and Python paths. Keep the accepted #9947 follow-up outside this PR scope.
…ffix relabels

Compaction no longer relabels untouched trailing fragments, so a recorded
group always records the live rows of the fragments it rewrote. Drop the
metadata-only relabel case from history validation, and update tests that
relied on relabels: disjoint recorded and unrecorded compactions now land
without conflict, and an index keeps covering fragments compaction left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added A-python Python bindings bug Something isn't working labels Oct 28, 2026
@wjones127
wjones127 marked this pull request as ready for review October 28, 2026 22:32

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Gate recommendation: approve with a non-blocking risk.

Restoring commit-time history validation protects address-based indices without restoring suffix relabels or their compaction conflicts.

Saved index metadata reattached after its translation history is cleaned up can still make queries miss rows. This path already exists on the base and remains tracked in #9947. Rebuild segments against the current dataset after cleanup; no additional admission change is requested here.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. K-risk Latest Gatekeeper recommendation includes a non-blocking risk. labels Oct 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-python Python bindings bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance. K-risk Latest Gatekeeper recommendation includes a non-blocking risk.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants