Skip to content

fix(transform): sort union_no_overlap inputs by timestamp - #838

Merged
ErikBjare merged 1 commit into
ActivityWatch:masterfrom
TimeToBuildBob:fix/union-no-overlap-sort-inputs
Oct 26, 2026
Merged

ErikBjare merged 1 commit into
ActivityWatch:masterfrom
TimeToBuildBob:fix/union-no-overlap-sort-inputs

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

union_no_overlap walks both lists in ascending timestamp order, but query_bucket returns events newest-first, and the query function passes its arguments through unsorted. The result then contains overlapping events and durations are double-counted.

Repro against a v0.14.1 --testing server: window and AFK buckets that each fully cover the same 40 minutes.

union_no_overlap(query_bucket("w"), query_bucket("afk"))                                  -> sum_durations 4200.0
union_no_overlap(sort_by_timestamp(query_bucket("w")), sort_by_timestamp(query_bucket("afk"))) -> sum_durations 2400.0

aw-webui hits this path with raw bucket output:

  • canonicalEvents: stopwatch_events = query_bucket(...) then union_no_overlap(stopwatch_events, events)
  • activityQuery: not_afk_curr = filter_keyvals(query_bucket(...), ...) then union_no_overlap(not_afk, not_afk_curr), once per AFK bucket. With two or more devices the total is inflated.

The fix sorts both inputs inside the transform, as period_union and filter_period_intersect already do. Inputs that are already sorted produce the same output as before.

Test: newest_first_inputs_are_sorted_before_merging feeds reversed lists. It fails without the sort (six overlapping spans) and passes with it. cargo test -p aw-transform -p aw-query passes and cargo fmt --check is clean.

aw-core's Python union_no_overlap documents sorted input as a precondition and has the same exposure, so it needs a matching change.

query_bucket returns events newest-first, but the merge walks both lists
in ascending order. Unsorted input emitted overlapping events: a 40-minute
span with full coverage in both lists summed to 70 minutes. period_union
and filter_period_intersect already sort their inputs.

Git-Session-Id: f67bd883-b10a-49dd-99de-2938d082f464
@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This PR appears safe to merge.

Summary

union_no_overlap now sorts both inputs before merging, so newest-first events no longer cause double-counted durations.

  • The event-union transform sorts both lists before merging.

Reviews (1) · Last reviewed commit: "fix(transform): sort union_no_overlap in..." · Reviewed by Greptile

@TimeToBuildBob

TimeToBuildBob commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 AI code review

In aw-transform/src/union_no_overlap.rs, union_no_overlap now calls sort_by_timestamp on both inputs before creating the merge iterators, and its documentation describes this ordering step. The new newest_first_inputs_are_sorted_before_merging test compares reversed inputs with ascending inputs, checks union invariants, and asserts a total duration of 40 minutes.

Safe to merge — no P0/P1 findings

Confidence 5/5

✅ No thread-worthy findings. Advisory notes follow; they are retained without opening review threads.

1 advisory finding (summary-only, not scored)

These P2 guard, heuristic, trade-off, or documentation claims are retained for judgment without opening review threads.

⚠️ P2 medium — aw-transform/src/union_no_overlap.rs

This is a fix(...) PR but no test files are included in the diff. Erik's feedback: 'where is the repro & fixes they are supposed to catch' (gptme#3441), 'that measurement should come with a regression test' (gptme#3446). Add a test that would have caught this bug. (Advisory: Erik merged all such PRs but consistently requested tests.)

Add a test file that reproduces the bug before the fix and passes after it.

How this was verified: static preflight: fix-commit + touched-files scan (rule 7)

Files changed (1) — the diff as I read it
  • aw-transform/src/union_no_overlap.rs — Imports sort_by_timestamp, sorts both merge inputs, documents the ordering step, and adds a regression test using newest-first event lists.
Previous review passes
commit score findings engine when
15cdf1a86f72 5/5 0 subscription 2026-10-26 04:42 UTC

Reviewed 15cdf1a86f72 · panel: codex:gpt-6.1-sol + claude-code:sonnet + grok-build:grok-4.6 · subscription engine · 16s · about this reviewer

Maintainer commands

@TimeToBuildBob review (own line) — fresh review · @TimeToBuildBob fix — a worker acts on the findings. Once per comment; 👀 = received.

The optional OpenRouter verifier was unavailable. Findings retain their original severity.

@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.42%. Comparing base (656f3c9) to head (15cdf1a).
⚠️ Report is 219 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master     #838       +/-   ##
===========================================
+ Coverage   70.81%   85.42%   +14.61%     
===========================================
  Files          51       88       +37     
  Lines        2916    12613     +9697     
===========================================
+ Hits         2065    10775     +8710     
- Misses        851     1838      +987     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

The missing-test advisory is a false positive: this diff adds the inline Rust regression test newest_first_inputs_are_sorted_before_merging in aw-transform/src/union_no_overlap.rs. No extra test file is needed. All eight checks are green, and Greptile reports 5/5. No code change needed; waiting for a maintainer merge (Bob has triage, not push permission). The matching Python fix is ActivityWatch/aw-core#185.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI-green and mergeable (Greptile 5/5) — waiting only on a maintainer click.

This PR is ready to merge, but the bot has pull-only access to this repo and can't self-merge — surfacing it here so it isn't lost. The monitoring loop will stop re-flagging it now that this note is posted.

ErikBjare pushed a commit to ActivityWatch/aw-core that referenced this pull request Oct 26, 2026
query_bucket returns events newest-first, but the merge walks both lists
in ascending order. Unsorted input emitted overlapping events: a 40-minute
span with full coverage in both lists summed to 70 minutes. period_union
and filter_period_intersect already sort their inputs.

Counterpart of ActivityWatch/aw-server-rust#838.

Git-Session-Id: f67bd883-b10a-49dd-99de-2938d082f464
@ErikBjare
ErikBjare merged commit c684083 into ActivityWatch:master Oct 26, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants