Skip to content

[Subscription] Add a switch for consensus fallback polling - #19062

Open
Caideyipi wants to merge 2 commits into
apache:masterfrom
Caideyipi:codex/subscription-poll-fallback-switch
Open

Caideyipi wants to merge 2 commits into
apache:masterfrom
Caideyipi:codex/subscription-poll-fallback-switch

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Description

PR #18738 lets consumers whose assigned regions are idle borrow ready events from regions assigned to other consumers. This can send events from one region to multiple consumers and reorder downstream processing in streaming window workloads.

Add the startup setting subscription_consensus_fallback_polling_enabled. It defaults to true to preserve existing behavior. Set it to false in DataNode configuration and restart the DataNodes to restrict polling to each consumer's assigned regions. Normal ownership reassignment remains unchanged.

Regression tests cover configuration loading, retaining the existing enabled fallback, disabling borrowing when assigned regions are empty, and disabling borrowing when a consumer owns no regions. A cluster IT uses one DataRegion and two consumers in the same group to verify that the unassigned consumer cannot borrow messages while the assigned owner still receives every row.

Validation

  • Spotless formatting and IT selection checks passed.
  • Clean targeted unit-test reactor run: 14 tests passed.
  • Clean Chinese-locale reactor test compilation passed.
  • Remote 1 ConfigNode / 3 DataNode cluster IT: 1 test passed, with no failures, errors, or skips. Subscription was temporarily enabled in an isolated verification source copy; the enablement override is excluded from this PR.
  • Negative control: enabling fallback caused the same IT to fail at the assertion that the unassigned consumer must not receive messages.

@Caideyipi
Caideyipi marked this pull request as draft October 30, 2026 07:23
@Caideyipi
Caideyipi marked this pull request as ready for review October 30, 2026 07:37
@Caideyipi
Caideyipi marked this pull request as draft October 30, 2026 07:54
@JackieTien97
JackieTien97 marked this pull request as ready for review October 30, 2026 08:24
@Caideyipi
Caideyipi marked this pull request as draft October 30, 2026 08:57
@Caideyipi
Caideyipi marked this pull request as ready for review October 30, 2026 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants