Skip to content

fix(barrier): handle generation counter wraparound on 32-bit targets - #413

Merged
tisonkun merged 4 commits into
apache:mainfrom
orthur2:fix/barrier-generation-identity
Nov 8, 2026
Merged

tisonkun merged 4 commits into
apache:mainfrom
orthur2:fix/barrier-generation-identity

Conversation

@orthur2

@orthur2 orthur2 commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Summary

On 32-bit targets, a Barrier breaks after 2^32 completed generations. Its generation is a usize advanced with += 1, and poll treats a woken follower as done only when the generation it joined is less than the current one, while Drop checks for any difference. I reproduced both failure modes on v0.7.4 by starting the counter at usize::MAX. Debug builds panic on the overflow, and every later generation's final arrival panics again, so no waiter is released after that. Release builds wrap to 0, and polling a follower released by the wrapping generation re-registers its stale waker token, which panics on a vacant slot or makes a next-generation follower that reused the slot miss its wakeup.

The first commit makes the counter a u64 advanced with wrapping_add, and poll and Drop now share one identity check, BarrierState::is_completed, the same shape Phaser uses for its phase number. Two new unit tests start the counter at u64::MAX, poll or drop a woken follower after the wrap, and check with a counting waker that the next generation's follower is still woken. The existing drop test in barrier_test now asserts that wake as well. Before, it passed even with the check in Drop removed.

The second commit is a separate optimization. If you'd rather land only the fix, I'll drop it. A follower's first poll currently takes the state lock twice, once in wait to count the arrival and once in BarrierWait::poll to register the waker. Both now happen in the future's first poll under one lock, and the wait future stays at 40 bytes on 64-bit targets.

Design Notes

The identity check alone makes the wrap harmless, but a woken follower left unpolled for a full cycle of the counter would match the current generation again. With a 32-bit usize that cycle is 2^32 generations, so the counter is 64 bits on every target, which on i686 grows the wait future from 20 to 24 bytes and Barrier itself from 40 to 44 bytes. At the roughly 60 ns per generation of reused_generation_fanout/1 below, a full u64 cycle would take about 35,000 years. I also tried removing the counter and keeping per-node completion state in a WaitList. That closes the full-cycle case, but it would replace the WakerSet storage to cover a state that needs 2^64 generations.

Benchmarks

main is cbdcd0f. #412 landed after it but doesn't touch Barrier. Apple M5, macOS 26.4, Rust 1.96.0. These are the cases of cargo x bench --bench primitives -- barrier, which poll the futures by hand without an executor. I ran binaries for main, the first commit and both commits in rotating order. Each cell is the median of 12 rounds, each round the median of 5 runs, in ms. Change compares both commits with main.

Benchmark main First commit Both commits Change
reused_generation_fanout/1 313 306 294 -6.1%
reused_generation_fanout/4 216 213 192 -11.1%
reused_generation_fanout/16 179 179 155 -13.4%
cancel_pending 332 335 314 -5.4%

On 64-bit targets the first commit only changes a branch condition, so its column shows noise.

Validation

cargo x test (696 passed), cargo x check (26 feature configurations), cargo x lint, and Miri on the Barrier unit tests and barrier_test, natively and for i686-unknown-linux-gnu.

A `Barrier` waiter kept the generation it joined. `Drop` treated the
waiter as completed once the current generation differed from that
number, but `poll` only once the number was less than the current one.
The counter was a `usize` incremented without wrapping, so completing
generation `usize::MAX` overflowed it. A debug build panicked on that
increment before waking anyone and left the counter in place, so the
final arrival of every later generation panicked the same way and no
waiter was released again. A release build wrapped to zero. The waiters
released by the wrapping generation still held `usize::MAX`, which is
never less than the current number, so polling one passed its stale
token to the drained waker set: it panicked if that slot was vacant, or
replaced the waker of the next generation's follower in the slot and
stayed pending, so that follower was never woken. On 32-bit targets
this takes 2^32 generations.

Complete a waiter when the generation it joined is no longer current,
in one check that `poll` and `Drop` share, the same shape `Phaser` uses
for its phase number. Comparing by identity makes the wrap itself
harmless, but a woken waiter left unpolled for a full cycle of the
counter would match the current generation again, and that cycle is
only 2^32 generations for a 32-bit `usize`. Use a 64-bit counter with a
wrapping increment so the cycle is out of reach on every target.

Add unit tests that start the counter at `u64::MAX`, poll or drop the
woken follower after the wrap, and check that the next generation's
follower is still woken. The existing drop test now asserts that wake
too; it passed even with the generation check in `Drop` removed.
`Barrier::wait` took the state lock twice on a follower's first poll:
once in the async fn body to count the arrival and once in
`BarrierWait::poll` to register the waker. Move the arrival into the
future's first poll so the count, the completion check and the waker
registration happen under one acquisition, and let the future return
the leader result itself. The leader can no longer complete a
generation between a follower's arrival and its registration.

An unset token marks a future that has not arrived yet, as the index
does in the semaphore's `Acquire`. This relies on the `async fn`
wrapper, which panics if polled after completion instead of letting the
inner future record a second arrival. The wait future stays at 40 bytes
on 64-bit targets and 24 bytes on i686, the same as with the separate
arrival.
@orthur2
orthur2 requested review from QwQBiG and tisonkun November 8, 2026 04:41
@tisonkun
tisonkun merged commit 452d858 into apache:main Nov 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants