Skip to content

Bound orphan task cleanup transactions - #3313

Draft
hanshengzhao wants to merge 1 commit into
moby:masterfrom
hanshengzhao:codex/3312-orphan-task-cleanup
Draft

hanshengzhao wants to merge 1 commit into
moby:masterfrom
hanshengzhao:codex/3312-orphan-task-cleanup

Conversation

@hanshengzhao

Copy link
Copy Markdown

Startup cleanup can fail when several orphaned tasks are deleted in one batch. Each delete action carries the full task object, and the batch size estimate is checked after adding an action; the combined Raft request can exceed MaxTransactionBytes and prevent orchestrator initialization.

Delete orphaned tasks in individual store transactions before the existing reconciliation batch. This keeps each proposal bounded while preserving the existing handling for service-backed and standalone tasks. The tradeoff is one Raft proposal per orphan during startup, so cleanup takes longer when a large number of orphaned tasks exists.

The regression test creates three individually valid large orphan tasks and verifies startup removes them while retaining unrelated tasks. It fails against the previous implementation because the combined proposal exceeds the Raft request limit.

Validation:

  • go test ./manager/orchestrator/taskinit -run TestCheckTasksLargeOrphanTasks -count=1
  • go test ./manager/orchestrator/... ./manager/state/store
  • go test -race ./manager/orchestrator/taskinit
  • git diff --check

go test ./... was attempted. The node package failed in TestManagerRespectsDispatcherRootCAUpdate with node did not ready in time; the same test fails on the unmodified base revision. The scheduler failure seen during the full run passed when rerun alone on both base and patch revisions.

Fixes #3312

Fixes moby#3312

Signed-off-by: hanshengzhao_dxm <hanshengzhao@duxiaoman.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delete orphaned tasks in size-bounded startup transactions

1 participant