Repository navigation
Conversation
A selector the evaluator reads returns every bucket of every series, about 140 bytes of JSON each. A 24 h Grafana panel over 1,000 series at a 60 s step is 193 MB of JSON and 350 MB of heap, far past a Worker's 128 MB. The common panel shapes now run in ClickHouse whole: a range function over a selector (rate, increase, delta, irate, idelta, resets, changes, the *_over_time family) and sum, avg, min, max, count or group by labels over one or over a bare selector. Each bucket fans out to the steps whose window holds it and the window is recombined with the evaluator's exact rules, so one value per series or group and step comes back. The source declines a pushdown it cannot make exact (more than one stored metric, an `le` matcher) and the evaluator reads the selector as before. On 12.7 M rows in chDB (scripts/bench-promql.ts): the 24 h, 60 s-step `sum by (status) (rate(...[4m]))` went from 193 MB of JSON, 351 MB of heap and 1,062 ms of evaluation to 0.1 MB, 30 MB and 2 ms; SQL time is unchanged at about 1.5 s. Every corpus query still equals the reference with pushdown on and off (apps/cli/test/promql-engine.test.ts), and a selector read past 24 MB now fails with a clear execution error instead of exhausting the isolate.
# Conflicts: # lib/promql/src/engine/index.ts # lib/promql/src/engine/plan.ts # packages/backend/src/services/promql/PromqlQueryService.test.ts # packages/backend/src/services/promql/PromqlQueryService.ts
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 9 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This comment has been minimized.
This comment has been minimized.
# Conflicts: # lib/promql/src/engine/evaluate.ts
A pushed-down subtree past the series cap now fails like a selector read does instead of returning a truncated answer. sum by (__name__) over a range function no longer carries the metric name, which every pushed function but last_over_time drops. Grouping by le on data that is not a histogram bucket is left to the evaluator. Both name cases join the compliance corpus: 420 of 420 match Prometheus 3.5.
This comment has been minimized.
This comment has been minimized.
# Conflicts: # packages/backend/src/services/promql/PromqlQueryService.ts
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
# Conflicts: # apps/cli/test/promql-engine.test.ts
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Maple review🟢 Confidence 10/10 · safe to merge. Window and bucket math is pinned by the chDB on/off parity corpus and the compliance queries; the open Part 3 of native PromQL:
Findings🔵 Note · F4 ·
|
|
Note ⏳ Maple is reviewing this pull request at |
Native PromQL, part 3: evaluate range functions and aggregations in ClickHouse.
Why
A selector the evaluator reads returns every bucket of every series, about 140 bytes of JSON each. A 24 h panel over 1,000 series at a 60 s step was 193 MB of JSON and 350 MB of heap, far past a Worker's 128 MB.
What changed
rate,increase,delta,irate,idelta,resets,changes, the*_over_timefamily) andsum/avg/min/max/count/groupby (...)over one or over a bare selector. Each bucket fans out to the steps whose window holds it, and the window is recombined with the evaluator's exact rules, so one value per series or group and step comes back.lematcher) and the evaluator reads the selector as before.Evidence
apps/cli/scripts/bench-promql.ts, 12.7 M rows on chDB (1,000 counter series and 200 histogram series, 15 s scrape over 24 h):sum by (status) (rate(...[4m]))sum by (route) (rate(...[5m]))histogram_quantileEvery corpus query still equals the reference with pushdown on and off (chDB), and more than 100 corpus subtrees were pushed.
🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.