[jsolemd]

Summary

• Adds the index-persistence seam from Add an index transaction seam for plugin-owned records and publication #4721. Every registered implementation runs, in registration order.
• apply runs inside the index write's transaction, so a raise rolls back the index rows and every earlier plugin's records. publish runs after the commit and the writer's own cleanup; its error reaches the caller and the commit stays.
• The writers share two helpers: apply_index_persistence where index rows are written (persist_pipeline_result, the resume checkpoint, init's final write after analysis, the step shared by incremental and docs-mode updates, and the health re-score), and publish_index_persistence as the last statement of each writer (init, index_repo_full, incremental and docs-mode updates, the health re-score, server index jobs). There's no default implementation, and with no plugin registered the writers behave as before.

Related Issues

Closes #4721.

Test Plan

The tests run init, index_repo_full, incremental and docs-mode updates and the health re-score on SQLite with two registered plugins, each for success, an apply failure and a publish failure. Separate tests cover the init checkpoint, init's final write after a successful checkpoint, and a server job against the real store with an apply or a publish failure. Each case fails with its writer's call removed.

• Tests pass (pytest tests/unit tests/providers): 40,053 passed.
• Lint passes (ruff check on the changed files).
• Web build passes (npm run build) (no frontend changes)

Checklist

• My code follows the project's code style
• I have added tests for new functionality
• All existing tests still pass
• I have updated documentation if needed (the plugin seam catalog row)

[RaghavChamadiya]

Thanks Jon, this is close to exactly what we discussed: every implementation through all(), apply inside the index transaction, publish after the commit, and one pair of helpers shared by the writers. A few things before we merge:

1. Give plugins the traversal set on full writes. IndexWrite carries result.parsed_files, but core deliberately prunes by the files the traversal found, not the files that parsed (see the comment in persist_pipeline_result around lines 164-168): a locked or unparsable file would otherwise look excluded and lose its rows. A plugin that prunes against parsed_files repeats that bug, so please add the traversal paths to IndexWrite and say in the docstring that parsed_files is not a presence list. traversal_complete=True is also hardcoded for full writes even when some files failed to parse.
2. The health re-score writer. _rescore in update_cmd/persistence.py (the exclude-pattern change path) deletes GitMetadata rows for newly excluded paths and commits without calling apply or publish, so a plugin's per-file records go stale. Please route it through the helpers too.
3. Field meanings. On incremental writes traversal_complete is ctx.reconcile_full_scope, which means "a full reconcile is happening", not "the walk finished". failures is always empty on full writes. And the resume checkpoint reports as "full", so a plugin can't tell a checkpoint from the final write. A checkpoint (or resume) mode, plus accurate names or docstrings for the other two, would cover it.
4. Tests. A real rollback test for the server job path (today it mocks persist_pipeline_result), and the init branch where the checkpoint already persisted the index, where the final transaction rewrites analysis and symbol complexities after apply has run.

Two small ones while you're there:

• The catalog row in PLUGINS.md says the final write retries after a checkpoint failure. That's only true when the checkpoint did not mark the index persisted, so it needs a small rewording. It's also worth one line saying a cleanup failure after the commit skips publish.
• The incremental and docs-mode step lists build tombstone_candidates even with no plugin registered. A guard on index_persistence.all() being empty keeps the no-plugin path the same as today.

Happy to look again as soon as you push.

[jsolemd]

Thanks for the review - made changes per your input:

• traversed_paths replaces traversal_complete. It's the set core reconciled against on that write, and None when core didn't reconcile. parsed_files is documented as not a presence list.
• deleted_paths now lists what the liveness prune actually removed.
• _rescore applies and publishes through the helpers.
• The checkpoint has its own mode, and init applies again after analysis.
• failures is now degraded.
• The server test runs the real persist_pipeline_result, and a new test covers init after a checkpoint.
• The catalog wording and the no-plugin guard are in.

I also merged main to resolve a conflict with #4393. Separately, re-running init keeps git_metadata and wiki_symbols rows for deleted files until the next update, on main too, so I'll open an issue for that.